Business intelligence (BI) professionals spend a significant amount of time wrestling with requirements gathering, specifications, data quality issues and other essential day-to-day tasks. The job naturally requires a lot of attention to detail, which can sometimes result in a loss of focus on the bigger picture.
The ultimate goal of BI is to inform, educate and add business value by telling stories about numbers. If it all sounds like the 1999 hit movie "The Matrix", where humans can 'picture' events and stories by reading a scrolling green screen of code, then that's because it's an accurate analogy. Numbers are just stories about people.
Scotland recently voted in a referendum on whether or not to leave the over 300-year-old United Kingdom. The vote was 45% 'no' versus 55% 'yes'. With a large turnout, this is a comprehensive measure of almost the entire population of Scotland's opinions about independence. It's not just a number. The polls leading up to the result were actually more evenly split, with predictions almost as close as 50/50, although the 'yes' vote never became a clear favourite. One individual even bet £900 000 (approximately R16 million) on the outcome. As he says: "I'm a bit of a data geek and information nerd." As it turns out, he had a winning bet with 'no', winning around £200 000 (around R3.6 million). With the polls overstating the closeness of the result, it must have made for a few uncomfortable moments. So, why the difference?
Having been born in Scotland myself, I understand the dilemma - the head votes 'no' but the heart favours 'yes'. Throughout the campaign, the 'yes' campaigners were more vociferous, visible and passionate. 'No' voters were a little less enthusiastic about stating their preferences, as they still felt the emotional pull of a 'yes' vote; this could help explain the polling discrepancies. Emotions were affecting the accuracy of the numbers.
Noise pollution
The American Nate Silver is famous for predicting 49 of 50 states' outcomes in the 2008 presidential elections, often disagreeing with existing pollsters and news pundits whose numbers were skewed by 'noise' and political bias. Poll numbers are as much a reflection of the people administering the poll as a reflection of the people taking the poll. The numbers can tell both stories.
I had a similar experience at the 2014 ITWeb BI Conference, again over a topic that provokes quite strong emotions in Gauteng right now, namely, e-tolling. I asked the audience a question: "Who here has an e-tag?" and around 30% of the audience put up their hands. I then specifically asked the opposite: "Who doesn't have an e-tag?" This time around, 60% of people put up their hands quite quickly and decisively. I suspect the missing 10% would tend to have an e-tag, but are hesitant to state so publicly. In discussions after the presentation, it was clear that having an e-tag often meant lukewarm support of e-tolling (at best), but not having one usually meant strong opposition and resistance to the concept. In this particular case, sampling the non-tagged gave a more accurate measure of the tagged percentage, whereas the reverse does not hold true.
BI professionals would be well advised to look beyond the obvious.
From the above it can be seen that, despite the old saying of 'the numbers don't lie', numbers themselves are both about people and created by people, and are not black and white, but usually subtle shades of grey. Sales departments may have a tendency to be 'generous' in determining what actually qualifies as a sale; marketing departments may deal in hard-to-measure concepts such as loyalty or brand affinity. Financial departments are probably closest in terms of a straightforward numbers approach, but as the recent global financial crisis proved, even accountants may be encouraged to 'massage' the numbers to lean one way or the other.
Regardless of the particular environment, BI professionals (or those who aspire to be) would be well advised to look beyond the obvious, in terms of requirements documents, technology choices and report specifications, and find the human story in both the numbers being produced and the consumption thereof.
The saying goes that "to someone with a hammer, every problem looks like a nail". Broadening the scope of day-to-day BI tasks to include the people aspect leads the way down interesting avenues. You may even find yourself betting on an election result one day, or the odds of the Gauteng e-tolls surviving until the next election. I know where I would put my money...
Share