This is the second article in a series of three on data fallacies—the most common mistakes in interpreting data that, far from helping us, can lead us to wrong decisions.
In the first one, we discussed the trap of summarized metrics—you can read it here; today we explore an even more dangerous one because it disguises itself as perfection: overfitting.
When data “learn by heart”
Imagine someone studying for an exam by memorizing all the questions from previous years. On test day, they might get a perfect score—but if the questions change, they won’t know how to respond. The same thing happens with data models that fall into overfitting: they look flawless because they fit the past with absolute precision, but they fail when faced with new situations.
In other words, data learn by heart, not the lesson.
Everyday examples
Weather prediction: A model that fits a single month in great detail may seem accurate. But the following month, patterns change and the prediction loses all validity.
The overly confident GPS: A GPS that memorizes every pothole and traffic light on your daily route will be perfect… only on that route. If you travel to another city, it becomes useless.
In People Analytics
In the world of people management, overfitting appears more often than we think:
-
A model that perfectly explains the results of a survey… but only for a specific period or area. When applied in another context, it fails.
-
A turnover indicator so specific that it only reflects particular cases and does not help anticipate real trends.
-
Performance predictions with too many irrelevant variables, resulting in fragile models that are impossible to replicate.
The problem is that apparent precision misleads us: we believe we have the “perfect model,” when in reality we have built a house of cards.
Final reflection
Overfitting reminds us that the past does not guarantee the future. In People Analytics—and in any data-driven discipline—we are not looking for explanations that fit perfectly with what already happened, but for robust models that allow us to anticipate what is coming.
Sometimes, perfection in data is not an achievement, but a trap.
By Yoel Kluk, Partner at Olivia Mexico.