AI Path Finder
Free consultation

Lesson · ⏱ 8 min read

🐼 Pandas: working with tables of data

EnglishHinglish

🏠 Real-life example

If you've used Excel, you already know the idea: rows, columns, filters and totals. Pandas is Excel inside Python. The difference is that it handles millions of rows, never crashes on big files, and repeats your steps exactly every time.

💡 In simple words

Pandas is the standard Python library for tabular data. A DataFrame is a table, and each column is a Series. With a few lines you can load a CSV, look at the first rows, filter, sort, fill in missing values, and group data to calculate totals or averages. Before any ML model can learn, data is almost always cleaned and shaped in Pandas. NumPy, which Pandas is built on, handles fast maths on arrays of numbers.

🪜 Step by step

  1. Load data: df = pd.read_csv("sales.csv"). A DataFrame is a table: rows and named columns.
  2. Look before you analyse: df.head(), df.shape, df.info() (types and missing values), df.describe() (summary numbers).
  3. Select and filter: df["city"], df[df["amount"] > 1000], df[["city", "amount"]].
  4. Clean: df.dropna(), df.fillna(0), df.drop_duplicates(), pd.to_datetime(df["date"]).
  5. Summarise: df.groupby("city")["amount"].sum().sort_values(ascending=False). This one line answers many business questions.

🔑 Key terms

DataFrame
A table with rows and named columns.
Missing value (NaN)
An empty cell. Must be handled before modelling.
groupby
Split rows into groups and summarise each, e.g. sales per city.

👀 See it in action

python
import pandas as pd

df = pd.read_csv("orders.csv")     # city, item, amount
print(df.head())                     # first 5 rows
df = df.dropna(subset=["amount"])   # drop rows with no amount
by_city = df.groupby("city")["amount"].sum().sort_values(ascending=False)
print(by_city.head(3))               # top 3 cities by sales

Load → clean → summarise. The everyday loop of data work.

⚠️ Common mistakes to avoid

  • Not checking data types: numbers stored as text ("1,200") won't add up. Clean and convert with pd.to_numeric.
  • Looping row by row with for. Pandas operations on whole columns are much faster.
  • Dropping missing values without thinking. You may delete most of your data or bias it; first check how many and why.

✅ Quick check

You want total sales for each city. Which Pandas tool fits?

  1. groupby()
  2. read_csv()
  3. head()
Show answer

A. groupby(): groupby splits rows by city, then sum() adds up the sales in each group.

🙋 Questions learners ask

Pandas or SQL: which should I use?

Both. Use SQL to pull and aggregate data inside the database, then pandas for exploration, cleaning and ML preparation in Python.

My file is too big for pandas. What now?

Read only needed columns (usecols), read in chunks (chunksize), or use tools built for big data like Polars, DuckDB or Spark.

📌 Remember

DataFrame = table. Most AI projects start with load → clean → groupby.

Learn this as part of a full course

Track your progress, take the quick checks and follow the lessons in the right order. This lesson is part of:

AI Foundations Path · ML Engineer · Data Engineer (AI focus)

Continue in the course