Imagine you build a model to detect a rare disease that affects one person in every thousand. You test it and it reports 99.9% accuracy. Impressive — until you look closer and realize the model simply predicts “healthy” for everyone. It never catches a single sick patient, and yet the number on the report looks spectacular.
This is the single most common trap in machine learning evaluation, and it is why precision and recall exist.
Start with the confusion matrix
Every classification metric is built from four counts. For a model that answers yes or no, each prediction falls into one of these boxes:
| Actually positive | Actually negative | |
|---|---|---|
| Predicted positive | True Positive (TP) | False Positive (FP) |
| Predicted negative | False Negative (FN) | True Negative (TN) |
A helpful way to remember the two error types: a false positive is a false alarm, and a false negative is a miss. Almost every real-world decision about a model comes down to deciding which of those two hurts more.
The three metrics, in plain words
- Accuracy = (TP + TN) / everything. Of all predictions, how many were right?
- Precision = TP / (TP + FP). When the model says yes, how often is it correct?
- Recall = TP / (TP + FN). Of everything that was actually positive, how much did the model find?
Notice what precision and recall have in common: neither one uses true negatives. That is exactly why they survive the rare-disease trap. In our example, the model that predicts “healthy” for everyone has zero true positives, so both precision and recall collapse to zero — which is an honest description of a useless model.
A worked example
Say a spam filter processes 1,000 emails. 100 are genuinely spam. The filter flags 80 messages, and 60 of those really were spam.
- TP = 60 (correctly flagged spam)
- FP = 20 (legitimate emails wrongly flagged)
- FN = 40 (spam that slipped through)
- TN = 880 (legitimate emails correctly left alone)
Precision = 60 / 80 = 75%. Recall = 60 / 100 = 60%. Accuracy = 940 / 1000 = 94%.
The 94% sounds great and tells you almost nothing. The two numbers that matter are that one in four flagged messages was innocent, and that four out of ten spam messages still reached the inbox.
The trade-off is real and unavoidable
Most classifiers do not output a hard yes or no. They output a probability, and you choose a threshold — say, flag anything above 0.5.
Move that threshold up and the model becomes conservative: it only says yes when very confident. Precision rises, recall falls. Move it down and the model becomes eager: it catches more real positives but also more false alarms. Recall rises, precision falls.
You cannot maximize both by tuning the threshold alone. Improving both at once requires a genuinely better model — better features, better data, better architecture.
Which one should you optimize?
The answer comes from the cost of each mistake, not from statistics.
| Situation | Costlier error | Prioritize |
|---|---|---|
| Screening for a serious illness | Missing a sick patient | Recall |
| Spam filtering | Deleting an important email | Precision |
| Fraud detection on transactions | Letting fraud through | Recall |
| Automatically banning user accounts | Punishing an innocent user | Precision |
| Search results on page one | Showing irrelevant results | Precision |
| Legal document discovery | Missing a relevant document | Recall |
Notice the pattern: when a human reviews the flagged cases afterwards, you can afford lower precision, because a person filters the false alarms. When the model acts automatically with no review, precision matters enormously.
The F1 score, and when it misleads
Sometimes you want one number. The F1 score is the harmonic mean of precision and recall:
F1 = 2 × (precision × recall) / (precision + recall)
The harmonic mean is used instead of a simple average for a good reason: it punishes imbalance. A model with 100% precision and 2% recall has an arithmetic average of 51% but an F1 of about 4% — a much more honest summary.
The catch is that F1 assumes precision and recall matter equally. In the disease-screening example they clearly do not. When one side matters more, the F-beta score lets you weight recall higher (beta greater than 1) or precision higher (beta less than 1).
Curves tell you more than points
Because everything depends on the threshold, a single metric is a snapshot. Two curves show the whole picture:
- The precision–recall curve plots the two against each other across every possible threshold. It is the better choice when positives are rare.
- The ROC curve plots the true positive rate against the false positive rate, summarized by the area under the curve (AUC). It is intuitive and widely used, but it can look flattering on heavily imbalanced data, because the huge pool of true negatives keeps the false positive rate small.
What to do in practice
- Look at your class balance first. If positives are under roughly 10% of the data, treat accuracy as decoration, not evidence.
- Write down the cost of each error type before training — ideally in money, time, or risk, not in abstract points.
- Always inspect the confusion matrix, not just the summary metrics. The raw counts often reveal a pattern the ratios hide.
- Tune the threshold as a deliberate decision, not by leaving it at the default 0.5.
- Evaluate on data that looks like production. If positives are rare in the real world, do not rebalance the test set — rebalance training if needed, but let the test set stay honest.
Conclusion
Accuracy is a comfortable number. Precision and recall are useful ones. They force you to answer a question that no algorithm can answer for you: which mistake are you willing to make more often? Once that question is settled, the metric to optimize becomes obvious, and so does the threshold.
If you want to go deeper into model evaluation, data preparation and the fundamentals behind these techniques, the free Artificial Intelligence and Machine Learning courses on Cursa are a solid place to continue.
