Lesson · ⏱ 8 min read
📏 Measuring models: accuracy, precision and recall
EnglishHinglish
🏠 Real-life example
Airport security has two kinds of mistakes: stopping an innocent traveller (a false alarm) or missing a real threat. A scanner that never beeps is '99.99% accurate', because almost nobody is a threat, but it's useless! Precision asks: when it beeps, how often is it right? Recall asks: of all the real threats, how many did it catch?
💡 In simple words
Accuracy, the share of predictions that are correct, is misleading when one class is rare, like fraud or disease. Always compare against a baseline, such as 'always predict the most common class'. Precision = correct positives ÷ everything you flagged. High precision means few false alarms. Recall = correct positives ÷ all actual positives. High recall means few misses. F1 combines both. Which matters more depends on the cost of each mistake: for cancer screening, recall is critical. For spam filters, precision matters so real emails aren't hidden. For regression, use errors like MAE (average absolute error).
🎬 See how it works
Out of 1,000 payments, 10 are fraud. The model flags 20 and 8 of those are real. Precision looks across the 'flagged' row, recall down the 'actually fraud' column.
🪜 Step by step
- Build the confusion matrix: true positives, false positives, false negatives, true negatives.
- Compute precision = TP ÷ (TP + FP) and recall = TP ÷ (TP + FN).
- Ask the business which mistake costs more: a missed fraud, or a blocked genuine customer?
- Move the threshold (0.5 is not sacred) to get the precision and recall balance you need.
- Report the chosen metric on the test set, compared with the baseline.
🔑 Key terms
- Precision
- Of everything flagged, how much was truly positive.
- Recall
- Of all true positives, how many were caught.
- Confusion matrix
- A 2×2 table of correct and wrong yes/no predictions.
👀 See it in action
1,000 transactions, 10 are fraud.
Model flags 20; 8 of them are real fraud.
Precision = 8 / 20 = 40% (many false alarms)
Recall = 8 / 10 = 80% (caught most fraud)
Accuracy = (8 + 978) / 1000 = 98.6%
Baseline 'never fraud' accuracy = 99% ← beats the model on accuracy!Accuracy hides the story. Precision and recall reveal it.
⚠️ Common mistakes to avoid
- Celebrating high accuracy on imbalanced data. Check what the 'always no' baseline scores.
- Picking a metric the business doesn't care about. Translate it: 'we catch 8 of 10 frauds and wrongly block 1 in 500 customers'.
✅ Quick check
For a cancer screening model, which mistake is usually worse?
- Both are equally fine
- Low recall: missing real cases
- Low precision: some false alarms
Show answer
B. Low recall: missing real cases: A missed cancer case can be life-threatening, while a false alarm leads to another test. So recall matters most.
🙋 Questions learners ask
What is ROC AUC vs PR AUC?
Both summarise performance over all thresholds. For rare positives, PR AUC is more informative because ROC AUC can look great even when precision is poor.
Which metrics for regression?
MAE (average size of error), RMSE (punishes big errors more) and MAPE (error in percent). Pick the one that matches how errors hurt.
📌 Remember
Beat the baseline, and choose precision vs recall by the cost of each mistake.
📖 Words from this lesson (AI dictionary)
Learn this as part of a full course
Track your progress, take the quick checks and follow the lessons in the right order. This lesson is part of:
AI Foundations Path · ML Engineer · AI Research Engineer · AI Product Manager
Continue in the course