Johannesburg, 02 Aug 2019
There’s a thing that governments, corporations, small and medium businesses, even individuals, do: they take the easy path. And in doing so, they fall for the McNamara Fallacy.
The McNamara Fallacy started with a government mis-measure of success when Robert McNamara was the US Secretary of Defense during the Vietnam War. The metric used by the US government was “body count”, which is a chilling measure in the first place. However, they didn't measure chaos, destruction, economic impact, misery, and importantly, they weren’t looking at the mood of their own people. All of the above (except body count) were what ended the war.
The McNamara Fallacy deals with the use of measurements as a fallacious way of setting goals.
There are four steps in the McNamara process:
Step 1: Measure whatever you can readily measure.
Step 2: Ignore what you can’t measure.
Step 3: Think that if you can’t measure it, it can’t be important.
Step 4: Consider that if it’s not important, it doesn't exist.
Already you can see that it’s a fallacy. In a nutshell, the mistake is that quantitative measures are real and qualitative factors are unimportant, therefore don’t exist. There are numerous qualitative factors that companies, schools and governments don’t measure, but they assuredly are real and are clearly more important than what is being measured and reported.
In IT, we use SLAs (service level agreements) to define what will be the measure of a successfully delivered service. We use measures such as MTTR (mean time to respond – we do love our acronyms), and MTTS (mean time to service), ignoring the effect that an incident has on the business because it is, well, difficult to measure.
Even the “mean” in MTTR and MTTS is a cop-out – like a man who drowned in a river that had a mean depth of 50cm. Averages tell us very little, it’s the outliers we should be highlighting. We report the number of incidents resolved, and unresolved, where if we do our jobs right, by doing effective RCAs (root cause analyses), we should be reporting on the number of incidents that led to a permanent resolution so they can never happen again. Even the SLA is a problem, because it looks at cause and not effect. This is why you can have unhappy business users even when the SLA is met. Those outliers that we have ignored may have cost a fortune in revenue and reputation loss, but goodness me, we met the SLA.
Many companies are turning away from SLAs. Instead, they are going for flexible contracts, where the bare minimum of services is defined in the contract, and the relationship is mapped out in detail. The average business person knows that things will go wrong, and wants a partnership with IT to know it will do everything in its power to fix problems and minimise impacts on the business. With a true partnership, both parties have a vested interest in the success of the company, not in whether the IT department is within its SLA.
The McNamara Fallacy can be found outside IT service delivery. How does one measure the success of an IT strategy, for instance? By the percent of enterprise objectives addressed by the IT strategy? Because that’s one of the measures given by a reputable IT research group. This measure is weird. Firstly, who says that just because the IT strategy addresses enterprise objectives, that it’s the right IT strategy, or indeed that these are the right enterprise objectives? And secondly, because there are some things in the enterprise objectives that IT cannot address, and also, the IT strategy should necessarily have a portion that is about IT and not the enterprise. Trying to map one to the other is a fallacy, and trying to measure the percentage fit is an equal fallacy. But it is easy to measure. Also, CIOs have to game the system to show that IT initiatives are strategic. And so the politics and fabrications creep in. And the IT budget is constrained if they cannot show strategic alignment or pure ROI.
So, when you’re looking for a service provider, don’t look for them to provide an SLA. Instead, give some thought to a flexible contract that focuses on the relationship and on the partnership. Also, look for an interest in your business and its results. Most of all, don’t look for easy-to-measure success factors. They are usually only easy for the service provider, not for your business. If they measure the easy stuff, they’re wrong, but can prove they’re right.
Take-outs:
Quantitative measures, the easy ones, ignore the important areas of performance.
Averages are a misleading measure of successful service delivery.
Why do we have unhappy users, even when the SLA is met?
Focus on the partnership, not the SLA.
When you’re choosing a service provider, look for a relationship, not a contract.
Share