Have you also noticed how QUANTITATIVE our world has become? Common sense has been pushed to the back in favor of “objectification”. Don’t get me wrong here; I am not against measuring, but I am against measuring being the foundation.
WARNING: this is going to be a waaaay too lengthy post and I have no TLDR for you. At some point, this may very well just be the start of a book …
Measuring Quality
This post was triggered by a LinkedIN post from a “Head of Innovation” who asked his followers to count the number of items on his list of don’ts your company was guilty of. If you found more than 4 (out of over 30 …) you weren’t innovative. I jokingly asked if his post was part of his content-marketing calendar for that month which he actually confirmed … You know the kinda guy, right? I’m sure he had consultancy services ready if you met the (very low) bar.
So what is actually wrong with his question? By counting all 30+ topics with the same weight (=1), he threw all of them under the same bus. Taking two examples from his pile I’d say that “our strategy is reactive” gets a lot more points than “we don’t collaborate with universities“. The bigger picture here is that we sometimes are measuring things in numbers that we shouldn’t be measuring at all, especially when it comes to quality. For some of the items in his post, I would value them so high that just meeting that one was excluding yourself from being innovative. For others, I could count way more than 4 to make them conclusive enough for ‘the label’.
KPIs are corrupting themselves by nature
Everybody that knows me, knows that I have been vigilantly opposing KPIs. That because I truly believe they are corruptive. Here’s a classic double example from Belgian politics: in the 90’s we had a prime minister (G. Verhofstadt) whose KPI was “a budget in balance” which basically means no more expenses than income. He managed to do this year after year, putting up the good news show. What they failed to mention there, is that some of the strategic incomes to make that happen were not long-term but one-off deals. A lot of the Belgian patrimonium was sold-and-leased-back during that period, putting an even bigger mortgage on the future. A true classic example of après nous le déluge.
The dangers don’t lie in what you measure, they lie in what you don’t.
read ‘the broken window fallacy’ for more what is seen vs what is not seen in economics theory
These types of examples are always top of my head when I hear KPIs. Here’s a true classic in business: LEAD GENERATION. This is how lead generation is usually measured: for X amount of sales we had Y amount of leads in the funnel. So for 10x sales, we need 10x leads. WRONG! For 10x sales, you need to increase your ratio (leak) and decrease the time from lead to deal (lag). Focussing lead generation based on top-of-the-funnel multiplication is LAZY and COUNTERPRODUCTIVE.
Now, why do I call this corruptive? It’s corruptive in the way KPIs are implemented. Since the marketing leader is incentivized on lead generation, he’s going to focus on lead generation, not sales turnaround. When the sales’ KPI is “increasing revenue” she’s not going to care about margins and just do more deals on higher discounts. If they are incentivized on increasing the average deal price, they will focus heavily on selling waaaaay too much to their customer, not caring about whether or not that customer grows or even churns later on. I’ve even seen salespeople sell stuff the company would later have to credit to the customer just to meet their quarterly targets.
The TRUTH is temporary
Did you know that your gut has a brain of their own? Seriously, Google “Enteric Nervous System”. I’ve only known this for a couple weeks and it blew my mind (pun intended). Up to very recently, the truth was that we all have one brain. This new information has lingered around in some medical communities for long, but has only been accepted very recently as a common theory. And like that many truths have at some point been proven wrong. (Wikipedia: superseded theories)
“The truth is but a temporary representation of reality, constrained bt the questions we’ve asked.’
If there’s one thing you want to take away from this blogpost it’s this part. Accept that everything you know and hold true, is just one possible representation of reality, based on the currently known empirical evidence you have been exposed to. And it may be proven wrong anytime by asking other questions that were not asked before or by a re-evaluation of the outcomes. This is how we went from the sun revolving around the earth to the earth revolving around the sun. Or how we got to the Higgs Boson particle and quarks.
One controversial example from my side: there’s enough scientific evidence to prove climate change is human-made. This, today, is the truth. It’s extremely unlikely but there is a chance this may be proven wrong somewhere in the future. We should accept this may happen, but we should act today as it will not.
The (f)law of the averages
There are three basic concepts I want to demonstrate that are part of the way we misinterpret the results of our questions by averaging out the results. The first is are we even asking the right type of question? What is the right sample size that is representable? The first type of question is a binary question. Is something black or white? We know by now that something is usually grey so we tend to ask for a scale (think NPS, the Net Promotor Score). Here’s what’s wrong with NPS or any other averaging out calculation:
If you’d average out this picture above, you’d get a perfect 50% of grey. Little did you know not a single block is actually 50% grey … So asking whether the picture was black/white or even how grey would not have given you the right result. What you totally missed was that this picture has a multimodel distribution, bimodal in this case, with peaks at 20% and 80%. This is how the data really looks like …
The second example is very simple: if the average depth of a river is 3ft, you’ll still drown in that 18ft hole right in the middle of it … Averages by nature tend to exclude outliers!
The 3rd example is my favourite! In the 1950s, lt. Gilbert S. Daniels measured up over 4000 fighter pilots in order to build the best suitable cockpit. Bare in mind, this sample size of the population is already skewed by the type of men that gets through the physical tests! They measured a dozen different points. Can you guess how many pilots actually fit within the total averages of all measurements, given even a 10% fault margin? NONE !!! NOT A SINGLE ONE!!! Every person is unique, every story is unique, every company is unique. If you are using other companies’ averages as your guideline without taking your context into account, you will fail sooner or later and you’ll probably blame the data!
The fact that we are all unique, is what makes us all the same!
ME again, and one of my life mottos!
So … what’s next then?
The first step is to always question the unquestionable. Everybody knows the famous meme “when you assume, you are making an ass out of U and me.” Go back to the first principles, reduce every concept to the unquestionable components. Well if you’ve ever been in a meeting with me, you’ll know I’ll even question everything you held unquestionable. In philosophy, this is called Cartesian Doubt. DOUBT EVERYTHING. That’s where you’ll find the Earth’s rotation path.
Secondly, change the measurement paradigm and move the questions back to QUALITY rather than quantity. Start with a common-sense question, follow up with a measurement where necessary, and follow up with a common-sense check. Don’t just start with asking what needs to be measured but what needs to be achieved. A business plan’s objectives is just an output, not an outcome.
- Are we doing the right thing? Is my team happy, rather than how much are our employees performing?
- How can we do better?
- What is our next step to better ourselves?
And when you are measuring, ask yourself:
- is every data point equal in value
- how are they distributed/clustered?
- what am I not measuring?
And BRING BACK THE GUT FEELING! The fact that it has a brain of its own says a lot 😉