The Missing Bullet Holes

The nation's response to the COVID-19 health crisis has been compared to our being on a war footing, with both public institutions and private firms mobilizing fully to defeat a common enemy.  The analogy is apt in many ways, particularly so as we consider the personal sacrifices being made daily by many of our healthcare workers, people serving in vital areas such as food and medical support functions, those in transportation and delivery operations and so many others.

War of a certain kind it is, and such times draw out both the best and worst in our human nature, whatever the cause or duration of the crisis. In this particular circumstance, I've heard it framed that any bet against our future robust recovery as a nation and as an economy is a bet against the exceptionalism of our science. In large part, that is true. It would also be a bet against capitalism and against the ability of our entrepreneurs, corporate leaders and public servants to eventually do the right things in terms of mitigating long-lasting health and economic damage for the benefit of our system and our citizens. I would never be willing to make that bet.

Part of the story of American exceptionalism and the power of why our system works the way it does comes from our inherent meritocracy: the best ideas and most talented among us rise. In the COVID-19 fight, private enterprise is increasingly stepping up to help. On the medical front, the Gates Foundation is involved. So are pharmaceutical firms based here and abroad including Johnson & Johnson, Merck, Sanofi, Novartis, Pfizer, Gilead, Moderna, GlaxoSmithKline, Regeneron and countless others. IBM has tapped their largest supercomputers. On the supply side we see Ford, GE, GM, Philips, Alphabet, Amazon, Apple, 3M, Anheuser-Busch InBev, and the list goes on and on.

Sometimes we're a little slow out of the gate -- as we arguably were in World War II -- but when we hit the gas and really mobilize, we do it for impact. Though the enemy is of a different sort this time, there are some similarities to the greatest generation's war. That parallel got me reading this week about Abraham Wald, W. Allen Wallis and the Statistical Research Group (SRG) at Columbia University, which started its work in the summer of 1942. Allen Wallis led the group, which consisted of 18 top statisticians, in their assignment of providing statistical support of various sorts for the war effort. In Wallis's words, it was "the most extraordinary group of statisticians ever organized taking into account both number and quality." The group made significant contributions to the field of statistics overall that reached far beyond any particular issue they addressed, and the range of their thinking was significant.

Abraham Wald, who came to the United States in the late 1930's because of concerns about political developments in Austria, became a key member of this group. One of the issues he tackled was exploring how best to protect aircraft from enemy fire. Weight of single-engine fighter planes was always a concern, and agility and durability created an ongoing tension in practical design. At the beginning of the war, the British Hurricanes and Spitfires were manufactured without added armor. But by the middle of 1940, they each carried additional steel plating to protect their most vulnerable sections. Part of Wald's work came into play in determining how armor on multiple types of Allied aircraft should be strategically placed.

The military originally felt it would make sense to examine the returning planes and catalogue which parts of their structures were hit most frequently with enemy fire, adding additional protection to those sections of the planes. Wald had a different view. After reviewing available data, he theorized that bullet hits were likely to be distributed more evenly in combat. That led to his unique insight: focusing on the bullet patterns on planes that made it back missed the point. The hits the surviving planes took were clearly not fatal; the flights that never returned had the bullets striking the most vulnerable parts of the aircraft. In other words, he focused on the missing bullet holes instead of those in clear sight.

Wald completed a complex series of calculations that included what levels of damage might prove fatal for various sections of a particular plane when encountering different levels of enemy fire. To significantly oversimplify his work, he recommended additional armor be placed primarily on sections where returning planes had not been hit, protecting the areas of the aircraft where damage was most likely to result in a plane's not returning home.

The military's initial idea made sense on the surface, but it's an example of the common thought pattern researchers call survivorship bias. Survivorship bias involves oversampling or overemphasizing the experience of a successful segment of a larger group to the exclusion or minimization of the rest. It results in an unwarranted bias toward outperformance and an underreporting of failures. As another example, consider what happens with longer-term performance statistics for small cap stocks. Any reliable stats must include the impact of the companies that didn't work out as well as those that did: for all of the Microsofts and Googles out there that succeeded spectacularly, many new companies never made it.  

How can survivorship bias impact us as business leaders? One of the strategic risks we face is that of overemphasizing successful performance and trying to replicate it, whether in the form of product launches, customer wins, sales performance or overall success of business lines. Failure can teach us certainly as much, and often more than success. It is important to include the entire data set -- failures, too -- when evaluating performance and projecting our prospects into the future. We must always look at the other side of any ledger. It's true for risk management. It's true for creating great companies, great products and outstanding teams.  

I think one of the world's wisest men, polymath Charlie Munger, puts it most effectively when he quotes the algebraist Jacobi as saying when solving any problem, "invert, always invert."  Instead of looking only at what attributes a product must have to succeed, explore what characteristics it must have to fail. There's a wonderful book on this topic by Don Keough, former president of the Coca-Cola Company, called The Ten Commandments for Business Failure. His idea for building an outstanding business is to think hard about what would cause it to fail, and do the opposite.

Sounds simple, right? Just like seeing the missing bullet holes and gathering the right story from the facts before us, it may be simple, but it isn't easy. That's what makes the journey so rewarding over time.

Be safe in this period of uncertainty, and know that there are very smart people tackling COVID-19 who are, among other things, looking at solutions that won't work to uncover what ultimately will.

If you want to meet virtually and discuss these various ideas or learn more about how Riverfall Strategies could help your business, just shoot me an email at keith@riverfall.is. Stay well!