Mid-sized business intelligence and Microsoft Excel - What happens next?

Why is business intelligence (BI) currently so topical for business owners and executives in mid-sized organisations? If you are reading this and are familiar with the notions of data marts, data warehouses, metadata, OLAP, dashboards, scorecards, advanced visualisation, and other equally geeky BI dictionary terms, this won't appeal to you. For everyone else trying to make sense of why BI is such a hot topic at the moment, and what role Excel will play in the BI industry in future, this seeks to unpack some of the issues causing the winds of change to blow across the BI industry as we once knew it.

What is fuelling the surge of interest in BI in the mid-sized market, and what is it about Microsoft Excel that everyone has such a love/hate relationship with it? Why is there no clear answer as to whether Excel is something that could be used in a more structured and formal way in a company as a BI solution? The expression “Spreadsheet chaos” (normally prefixed with appropriately placed expletives) is well known without restriction to organisational size, geographical boundaries or intellectual capacity of an organisation. I bet even Nasa, where the real rocket scientists work, also suffers in a sea of spreadsheet disarray.

For commercial reasons, the traditional BI vendors don't use Excel as a front-end delivery platform, obviously opting to develop their own proprietary information delivery platform, but still typically provide an Export to Excel function, which just serves to create further spreadsheet anarchy. So when all the skills are accounted for, all the licensing arrangements are in place, all the project teams have celebrated their successful sign-offs - it all ends up in Excel anyway!

As the renewal bill arrives from the BI vendor there is a sudden and collective realisation that the user has no access to author reports (and actually has no idea how to use the interface anyway), the solution needs a tweak to better fit user requirements, on the back of their success the key project stakeholders have resigned to move on to bigger and better BI projects, and the whole vicious cycle starts again. Do we have to suffer this unruliness for eternity or are there solutions to create some order?

Let's use accounting software applications as an example. In the mid-sized space and across the world, this industry is characterised by a few large dominant players, and then a few smaller vendors that are typically territory-bound and hold a small part of the market. This trend is probably the same across payroll, point-of-sale, supply chain and many other transactional applications that have become largely commoditised. They have an enormous user base of people that use very powerful functionality to capture a varied mass of transactions. But these users have very limited or restricted visibility of this constantly swelling entanglement of data they are creating. To compound matters the BI tool that was an after-thought in the development cycle of the accounting application is simply not cutting it, and these same users are getting restless and placing increasing demand on the vendors to take these applications to the next level, beyond the intentions of their original designs. There is a real need to meet the requirements of rapidly maturing mid-sized organisations hungry for data. The story it tells facilitates better decision-making to ultimately drive organisational performance.

While this mid-sized groundswell for BI grows, Microsoft keep releasing versions of Excel, each release a few steps closer to an “out the closet” relationship with BI, that makes it more powerful and more usable. But what's the point if most people are only using a fraction of the functionality available in earlier versions? Why keep upgrading a product that is only partially used and is only part of a total solution and possibly only perpetuating spreadsheet pandemonium?

Resistance to investing in further licensing of Excel, whether through new licensing or upgrades, is real and apparent. But let's be clear, there is also no lack of tales of BI projects falling over in a heap of unmet deadlines, unfulfilled promises, frustrated end-users, and a pile of burnt cash and, at the end of the day, Excel would have done very nicely as the only solution needed instead. The one clear fact that resonates across organisations and industries is that every business would like to create some order of this situation but doesn't necessarily have a clear strategy on exactly how it should make sense of the possibilities and begin working towards the end solution.

I guess it's also fair to ask if there are any right answers to the current issues facing BI and Excel, particularly regarding mid-sized businesses. Very recently the traditionally big players in the BI space all consolidated their efforts: Hyperion with Oracle, Business Objects with SAP, and Cognos with IBM. The spin doctors wrote of leveraging relationships to mine existing revenue possibilities and grow into the fast maturing mid-sized space. Often however the big players lack the flexibility and understanding of the mid-space and simply attempt to sell the same technology but at a greater discount. A bit like asking a toddler to drive a car - it just doesn't work, not because they don't have a license, but because they can't even reach the pedals. But even more recently, Sage, a global software icon in the mid-sized space, announced the discontinuation of Sage Intelligent Reporting, their own BI solution.

To sum up, on the one hand the big players are targeting mid-sized companies but on the other hand the mid-sized market owners are discontinuing their BI products. Talk about mixed messages from the market! All this talks to the relevance, or not, of BI in the mid-sized space and to be sure, the need for small and mid-sized businesses to have accurate and up-to-date essential reports at their fingertips is no different to their bigger counterparts in the commercial world. The difference being, in the absence of large IT budgets and specialised BI staff, a solution that offers immediate return on investment is crucial. Business decision-makers need new models with plug-and-play solutions and pay-as-you-go pricing models through which to obtain immediate access to trusted reports that provide real-time insight into their businesses

So what to do next? What are the options really and where does one start unravelling this to make a sensible call on the best solution for your business? As alluded to earlier, there is a school of thought that says that BI in the mid-sized space is dead and buried, which is a moot point. The more likely reality, however, is the existence of a burgeoning mid-sized market coupled with the emergence of the next generation of ambitious BI vendors willing to spend time to really understand this market to cater for the shifting profiles of the average user in this new and exciting market. One thing is for certain - Excel is not a BI solution as a stand-alone application, but it most certainly is a powerful decision support tool when integrated with the right applications and tools and when used within a structured framework.

Microsoft Excel has a growing reputation as a sexy application (if you can call a spreadsheet sexy) in the BI space precisely because it can give mid-sized businesses insight into their data. The face of BI is changing and Excel is become a central character. Users currently disengaged from BI will be empowered to leverage the benefits of BI because of the rapid integration of BI solutions into existing applications. The prudent option when seeking out a BI solution then would be to find the vendor that understands the specific market needs, has built the necessary connectors to the data, can remove data integrity issues and create a consistent flow of reliable data by using Excel as the output and distribution mechanism. Do these vendors exist? Absolutely, but don't look to the traditional BI market, look to the next generation for your solution.

Microsoft Excel is here to stay. Yes, similar predictions of immortality were made of, for example, the typewriter, and up to a few years ago nobody would have predicted the threat of fixed line telecoms becoming extinct in our new world, so maybe in 10 years time this prediction will be laughable and Excel will lie in the spreadsheet graveyard alongside those that have expired before. However, ask anyone that has worked with Excel for many years and, more relevantly in this context, talk to people in companies that have created BI technology that integrates with Excel to provide BI solutions for mid-sized organisations, and you will get a good appreciation for how relevant and critical Excel is to so many companies globally.

Excel is unlikely to lose its position of dominance in the world of BI and spreadsheets in a hurry. There are already, and will no doubt be many more, pretenders to the throne but Excel is pervasive, it is easy, and will get easier to use. It is immensely powerful and will get more powerful (despite everybody still only using a fraction of its functionality). With this anticipated growth in functionality and usability, it is easy to wager that Excel will remain a great BI base for the mid-sized market for a long time to come, and including it as part of your total BI solution would be a wise investment in technology and skills.

We all need to move on from the market perception, arguably created by marketing machinery of the large BI vendors, that using Excel in a BI environment is akin to putting lipstick on a bulldog. A combination of continued development from Redmond, with one eye on the next big BI opportunity, and the new generation of BI vendors that have grown up with mid-sized BI in their blood, means there are and will be very real, cost effective, simple and powerful BI alternatives for the mid-sized market. Quite possibly a topic for a separate debate, but let's not forget that the BI industry will certainly not be immune to the disruptive effect of fast-maturing software-as-a-service technologies and many BI flavours have already emerged in the cloud.

The future is in solutions that draw real-time data directly from a company's data source into pre-formatted Excel reports with built in drill-down capabilities and other one-click functionality to access the data. Such solutions provide low-cost integration into existing IT infrastructure while extending and strengthening its functionality and making the familiar Excel interface widely accessible as a BI tool in your business. The BI purists among us will shudder, but there is a good chance that the next generation will no longer have to be experts in BI to get decent information about their business. It may even become a case of - if you can do spreadsheets, you can do BI.

Join our Excel forum at http://www.alchemex-forum.com to join many like-minded report writers from around the world who are sharing their experiences and ideas on Excel in a BI context.