When Measuring Strategy Produces Statistically-Significant Rubbish

Regular readers of this blog will know that I am a fan of Roger Martin's writings on strategy and leadership. Martin's September 25, 2023, article in Medium, "Measuring, Managing, and Mattering," dismantles an idea that we often hear from corporate types on nonprofit boards—that good strategies always have hard quantitative metrics attached by which boards can track progress. Business school graduates are particularly fond of quoting Peter Drucker as saying, "If you can't measure it, you can't manage it," while challenging a head of school's more qualitative framing of goals. 

Martin reveals that Drucker is not the source of that quote (nor is he the source of his most famous supposed quote, "Culture eats strategy for breakfast"); rather, W. Edwards Deming wrote those words about measurement in a 1993 book. However, the whole quote, instead of a snippet, actually says the exact opposite of how it is typically used. But a misattribution is not the real problem Martin points out with an obsession with the measurement imperative.  

The bigger problem is that quantitative measurement forces a reductive analysis that can miss the big picture. In a quote worthy of becoming famous, Martin write, "Sure, the results will be statistically significant — but it will often be statistically-significant rubbish." As always when statistics are involved, the trick is in discerning the rubbish from the gold. Both can be equally shiny.