Data has become an essential tool for decision-making. However, even with abundant data, it's easy to fall into traps that lead us to mistaken conclusions.
These errors are known as data fallacies: flawed interpretations that arise from biases, simplifications, or inadequate ways of analyzing information.
In this three-part article series, we will explore some of the most common data fallacies and how they affect fields like People Analytics. Today we begin with the first one: the trap of summarized metrics.
In 1973, the statistician Francis Anscombe presented an example that remains relevant today: the Anscombe’s Quartet. These are four data sets that, when analyzed using summarized metrics like mean, variance, and correlation, appear identical. However, when graphed, they show radically different patterns.
This demonstrates a key point: numbers alone can hide completely different stories. Visualizing and contextualizing data is just as important as measuring it.
The Trap: Summarized Metrics in People Analytics
Summarized metrics are numbers that condense complex information into a single value, such as an average, a percentage, or a global index. They are useful for communicating results simply, but they can also lead us to make wrong decisions because they hide critical variations.
It’s like looking at a landscape through a blurred photo: you see the silhouette, but you lose the details that matter.
Examples to Understand It
1. The Average Temperature
Imagine you are told that the average temperature in a city is 68°F (20°C). Seems like enough information, right? But it's very different if the temperature is 86°F during the day and 50°F at night. If you rely only on the average, you might pack the wrong clothes and end up too hot or too cold.
2. The Average Travel Speed
Suppose you drive for one hour at 60 mph and the next hour at 20 mph. Your average speed will be 40 mph, but this number doesn't describe the reality: one stretch was fast and the other was desperately slow. If you only share the average, you are hiding a fundamental part of the experience.
Why It Matters in People Analytics
This error is especially common in People Analytics. We are tempted to summarize information into a single number:
-
"Our organization scored a 5 on leadership."
-
"We got an 89% participation rate in the survey."
These data points seem useful, but they don't tell the whole story:
-
In which areas is leadership strong, and in which is it weak?
-
Which groups didn't participate, and why?
When we only see the average, we lose the opportunity to design precise and meaningful actions. The true value emerges when we explore the variability and nuances behind the data.
Averages and other summarized metrics are tempting because they simplify communication, but they should not be the final destination of analysis. They are merely the starting point for asking deeper questions and discovering patterns that truly guide action.
In the world of data, the simple answer is not always the true one.
By Yoel Kluk, partner at Olivia.