The Perils of More Data

perils of more data

By: David Widmar and Sarah Hubbart

This Weekly Insights comes from the AEI Premium archives. 

Paul Slovic’s Study

Is more information the key to understanding?

A study on horse racing – of all things – might just challenge your assumptions.

Back in 1974, a psychologist named Paul Slovic gathered a group of seasoned horse racing handicappers to see how well they could predict the outcome of 40 horse races conducted over four rounds. Because there were 10 horses in each race, each handicapper’s bet could be expected to be right 10% of the time through random guessing alone.

As part of Slovic’s study, the participants received any five pieces of information that they wanted on each horse, including things like age, fastest speed, and the jockey’s experience or weight. Each handicapper was then asked to both predict the winner of the race and their confidence in that outcome. In the first round, with limited information (remember, only five data points), they turned out to be 17% accurate (Nearly twice as accurate as they should have been!) and their confidence in their pick was 19% (almost exactly right).

From there, Slovic increased the handicappers’ access to information about the horses. In round two, they received 10 pieces of information; in round three, 20; and in the fourth round, 40. However, the increased availability of data did not impact their accuracy. It remained at 17%. At the same time, their confidence nearly doubled, to 34%.

Understanding the Paradox

These results highlight an important lesson. While we may think that being better informed about a topic leads to better decision-making, in reality, additional information can lead to confirmation bias. Without realizing it, we seek to prove ourselves right.

It’s a paradox. More information can actually make us worse at decision-making. That’s because when we seek out information on a topic, it’s often because we are really just looking for data to back up what we already believe. It becomes very easy to give information that we agree with more weight than it really deserves.

Navigating Information Overload

We see this concept play out all the time in the world of agriculture. Maybe it’s when we’re looking at seed trial data. Social media adds a new dimension to the challenge. Are those Twitter conversations about the outlook for commodity prices really providing meaningful insights? Or are they just noise that instills a false sense of confidence?

Think back to the 2022 planting situation. Earlier this year, we might have wondered about the probability of U.S. farmers planting more than 89.5 million acres of corn, which is what the USDA initially estimated back in March. We likely would have started reading everything we could find about planting conditions in the Midwest and acreage reports. We might stumble across one article providing an estimate of total corn acres – and then another nine articles that seem to reinforce those results. But are those additional nine articles really adding more data worth considering? Or simply parroting the results you read earlier while minimizing the remaining unknowns?

Ask yourself: Am I really becoming more accurate through data or am I just fooling myself?

P.S. You could have read this – and many other ideas and articles – months ago on AEI Premium. If you’re wanting to improve your thinking and make better decisions, we invite you to start a free trial today to see what your neighbors, bosses, and colleagues have been tapping into.