An intelligent solution

The transformation of corporate data into business intelligence and knowledge has become surrounded by confusion. As a result of the transition from executive information systems to business intelligence (BI), there is confusion as to the difference between EIS and BI solutions. Executive information systems (EIS) have become synonymous with BI.

A data warehouse is a strategic solution that enhances BI and Online Analytical Processing (OLAP) technologies, where:

• Operational data stores use relational databases that are optimised for transaction processing.

• OLAP technology allows for data to be transformed to a structure that can be easily analysed.

EIS have been around since the late 1970s. In the early 1980s, a Harvard Business Review article entitled "The CEO goes Online" carried the story about the system that Ben Heineman used at Northwest industries. Extensive programming and expensive equipment characterised these early EIS systems. Decision support systems have also been around since the 1970s and were tools typically for the specialist analyst/programmer. Howard Dresner, of the GartnerGroup, coined the term "business intelligence" to describe a set of concepts and methodologies to improve decision-making in business through the use of facts and fact-based systems.

Data warehouse is not a technology, it`s a strategy.

Think strategic

According to Nigel Pendse of The OLAP Report, multidimensional analysis goes back to 1962 with the publication of Ken Iverson`s book, A Programming Language.

By 1970, a more application-oriented multidimensional product, with academic origins, had made its first appearance. By the mid-1980s, the term EIS had been born. By the late 1980s, the spreadsheet was already becoming dominant in end-user analysis, so the first multidimensional spreadsheet appeared in the form of Compete.

According to Cognos, until the arrival of the BI model, managers were spending 80% of their time analysing 20% of the effective and relevant data they had access to. With BI and best practices this could be reversed. It allowed them to spend 20% of their time looking at 80% of effective and relevant data - that data pertinent to their function or operation. BI was seen as a tool that made managers more effective because they were looking at relevant data rather than the irrelevant data they had been accessing in the past.

The maturation of BI saw the confusion over BI and EIS lift. But with OLAP now coming to the fore, the cloud of confusion is beginning to descend again. Until recently EIS and BI tools were plugged into operational systems where its use was limited by its inability to do large analysis of the relational and propriety data locked away in corporate databases.

Enter a new data structure that allowed data to be stored and processed in a multidimensional format. It was just the breakthrough the business market was waiting for to enable it to view and analyse data from whichever perspective it wanted. Used with BI tools OLAP truly helped to unlock data.

Relational database companies could not deliver OLAP functionality so new vendors such as Arbor and Redbrick, to name a few, emerged to fill the multidimensional database. And what a gap it was. The OLAP market grew by some 300% per annum. Oracle already had Oracle Express and the other traditional database vendors such as Sybase, Informix and IBM were not going to take this lying down. They quickly acquired and developed the required technology to bring out their own OLAP solutions.

Today, the OLAP market has become very crowded and there are very few, if any, database vendors supplying OLAP only. Casualties are inevitable. On the one side, the large database vendors, all now with their own OLAP offerings, operate from a position of strength in that their user base accounts for 80% to 90% of the world`s data.

On the other, you have the large BI vendors such as Cognos and Business Objects offering their OLAP solutions. They address a market still growing at 60% a year and have a foot in all camps. They have their own OLAP functionality, act as the front-end to the OLAP vendors and are a key component in the emerging data mining and data warehousing markets. Therefore rationalisation of the pure OLAP vendors in the middle must take place. Microsoft`s entry into the OLAP market with its Plato solution is bound to aggravate the situation.

Much of the confusion around data warehousing follows the market`s insistence that it is a technology. Data warehousing should never have been touted as a technology - and a proprietary one at that. It should rather be viewed as a business strategy with the flexibility to change and adapt to changing business requirements. It is a strategy that has to encompass BI and OLAP tools. These are essential to realising a quick return on investment, because they provide the flexibility to allow users to respond to changing times at the drop of a hat.

An emerging technology that will help ease the establishment of flexible warehousing is extraction, transformation and loading (ETL) tools from companies such as Ardent, Informatica and Prism (which now belongs to Ardent). Early data warehouse inflexibility was due largely to having to manually extract data from legacy systems. ETLs are able to automatically extract legacy and other data from wherever it might reside; transform, clean it and load it into the warehouse or data mart of choice.

Before implementing a data warehouse companies must decide whether or not they are looking for operational reporting solutions or management information reporting. If it`s the former, then the reporting and drill-down capabilities of most enterprise resource planning systems should suffice. If it`s the latter, then a data warehouse or mart is the answer.

Companies are still implementing data warehouses for the wrong reasons. But if these investments are correctly done, an organisation should see significant returns in a matter of weeks or months - not years.

According to the GartnerGroup: "Organisations that do not include end-user data access and user self-sufficiency in their goals and objectives within three years risk their firm`s competitive position." Furthermore, studies have shown that the return on investment on data warehouses and data marts that deliver business solutions exceeds 400%.

In future articles I`ll explain why there is no such a thing as a failed data warehouse. Rather there are expensive ones and those that ensure companies maximise their return on investments by using BI and OLAP.