Survivorship Bias — Abraham Wald and the WWII Airplanes

Survivorship bias is a tricky fal­lacy to deal with.

We’re all mes­mer­ised by suc­cess stor­ies. But study­ing those who are suc­cess­ful isn’t always the best idea.

You might suc­cumb to sur­viv­or­ship bias.
You might bet on flawed strategies.

Here goes:

Abraham Wald and the Allied Aircrafts

During World War II, the Allies stud­ied Nazi dam­age to their fight­er planes. Their study res­ul­ted in this dot­ted illustration:

The ana­lys­is of these dam­ages res­ul­ted in the idea that the Allied Forces should rein­force their fight­er planes in the areas show­ing the densest clusters of red dots. 

A stat­ist­i­cian by the name of Abraham Wald dis­agreed. Instead, he sug­ges­ted that they rein­force sur­faces with no red dots.

Wald was right, of course. By study­ing planes that some­how returned home, the red-dot­ted areas indic­ated non-fatal dam­aged regions. 

If it had been pos­sible to study all the planes shot down and des­troyed by the Nazis, the research team would likely have found an inverse pattern.

World War I and the Soldiers’ Helmets

In anoth­er example, we focus on a seem­ingly coun­ter­in­tu­it­ive phe­nomen­on that emerged dur­ing World War I. 

As mil­it­ary forces began to increase the issu­ing of hel­mets to sol­diers to pro­tect them from the per­ils of the bat­tle­field, an unex­pec­ted trend was observed: the num­ber of wounded sol­diers surged dramatically. 

At first glance, it may seem that the hel­mets were para­dox­ic­ally mak­ing it easi­er for more people to get hurt. The crux of this para­dox lies in the shift­ing dynam­ics of the cas­u­alty statistics: 

As hel­mets were intro­duced and dis­trib­uted more widely, they sig­ni­fic­antly reduced the num­ber of fatal­it­ies on the bat­tle­field. Soldiers who would have pre­vi­ously suc­cumbed to their injur­ies were now sur­viv­ing. Consequently, these sol­diers were clas­si­fied as “wounded” rather than “dead.”

This shift in cat­egor­isa­tion cre­ated the illu­sion of increased injur­ies when, in real­ity, the num­ber of fatal­it­ies had decreased. 

The Fallacy of Survivorship Bias

These examples under­score the import­ance of con­sid­er­ing the lar­ger con­text and being cau­tious about con­clu­sions based solely on super­fi­cial data.

Both stor­ies are excel­lent examples of a par­tic­u­lar fal­lacy — sur­viv­or­ship bias.

“Survivorship bias, sur­viv­al bias is the logic­al error of con­cen­trat­ing on entit­ies that passed a selec­tion pro­cess while over­look­ing those that did not. This can lead to incor­rect con­clu­sions because of incom­plete data.”
Source: Wikipedia

We often can’t help being enticed by suc­cess stor­ies in pop­u­lar cul­ture and busi­ness. But their stor­ies can lead us to fal­la­cious conclusions. 

Few think to study those who turned out to be unsuc­cess­ful. In terms of data sets, this is too bad since the unsuc­cess­ful typ­ic­ally out­num­ber the suc­cess­ful by a vast margin.

Read also: 58 Logical Fallacies and Cognitive Biases