By: David Widmar and Sarah Hubbart
“They sure don’t make ‘em like they used to…”
How many times have you caught yourself saying that? Because in the good old days, things just never broke down.
… or did they?
Survivorship Bias is the logical error of focusing on the people or things that made it past a selection process and overlooking those that did not. It is a form of selection bias that can lead to false conclusions.
Maybe you have an old tractor that still runs great. You might jump to the conclusion that the John Deeres of the ‘70s were better than today’s models. But because of the pressures of time and use, the only 1970s-era machines that have survived to the present day are the ones that were best built and maintained.
But, what about all the lemons? The tractors of yore that didn’t stick around? When we only focus on the few that made it through to today, we don’t get the full picture of what farm machinery was really like back in the ‘70s. Because no matter the decade, there is one universal truth: everything breaks down.
WWII provides another famous example of Survivorship Bias. Bomber planes were returning to base riddled with bullet holes. The army knew they needed to add armor to the planes to better protect them – but where? They analyzed all the planes that returned and noted that the damage was largely concentrated on the tail, body, and wings. Logic would indicate that these are areas that would likely be hit again.
But statistician Abraham Weld made an important observation – by only focusing on the damage of the planes that returned, they weren’t factoring in the damage on the planes that didn’t return. Those planes likely sustained damage to their engines. Once planes were hit in the engine, they went down and weren’t able to return to base. The tail, body, and wings were also vulnerable, but planes could sustain damage there and still make it back. So, the military should focus on improving the armor elsewhere.
It’s easy to fall victim to this way of thinking, of only considering the success of the “plane that survived.” This can make us feel like we understand the world more than we actually do. But we can’t ignore the importance of randomness.
To avoid Survivorship Bias, consider what you can learn from the examples that failed.
Did you enjoy this content? It was originally shared on AEI Premium, our network for decision makers in agriculture. Sign up for a live demo of AEI Premium hosted by Ryan Keller and Brent Gloy here: