Lesson · ⏱ 6 min read

🔗 Correlation vs causation, and sampling bias

🏠 Real-life example

Ice-cream sales and drowning cases both rise in summer. They're correlated, but ice cream doesn't cause drowning. Hot weather causes both. And if you survey only people at a gym about fitness habits, your results won't represent the whole city. That's sampling bias.

💡 In simple words

Correlation means two things move together. Causation means one actually makes the other happen. ML models are excellent at finding correlations but cannot, on their own, prove causation, so be careful with claims like 'feature X causes Y'. Sampling bias happens when your data doesn't represent the real world, and a model trained on biased data will make biased predictions. Before trusting any result, ask who is missing from this data and what else could explain the pattern.

🪜 Step by step

Correlation measures how two things move together, from −1 (opposite) through 0 (no link) to +1 (together).
Look for a third factor (confounder): umbrella sales and traffic jams rise together because of rain.
Check who is in your data (sampling bias): a survey of app users says nothing about people who uninstalled.
To prove cause, run a randomised experiment (A/B test): randomly split users so the only difference is the change you made.
When you can't experiment, be careful with words: say 'is associated with', not 'causes'.

🔑 Key terms

Correlation: Two variables tend to move together.
Causation: One thing directly causes the other.
Sampling bias: Data that doesn't represent the full population.

👀 See it in action

text

Finding: users who use the app's dark mode spend more money.
❌ 'Dark mode makes people spend more' (causation claim)
✅ Maybe power users like dark mode AND spend more.
   Test it: an A/B experiment that randomly switches dark mode on.

A randomised experiment is how you move from correlation to causation.

⚠️ Common mistakes to avoid

Concluding a feature works because its users are happier. Engaged users choose features, not the other way round.
Training models on biased samples, like only approved loans. The model never learns about people who were rejected.

✅ Quick check

A hiring model is trained only on past hires from one city. What's the risk?

It will be slower
No risk at all
Sampling bias: it may unfairly score people from other cities

Show answer

C. Sampling bias: it may unfairly score people from other cities: The training data doesn't represent everyone the model will judge.

🙋 Questions learners ask

Can ML models find causes?

Standard ML finds correlations useful for prediction, not causes. Causal questions need experiments or special causal inference methods.

Is a correlation of 0.3 weak?

It's modest, but it can still matter in business. Also, correlation only measures straight-line relationships; always plot the data.

📌 Remember

Correlation is not causation, and biased data gives biased AI.

📖 Words from this lesson (AI dictionary)

Learn this as part of a full course

Track your progress, take the quick checks and follow the lessons in the right order. This lesson is part of:

AI Foundations Path · ML Engineer

Continue in the course