Subscribe

Data warehousing - be realistic


Johannesburg, 11 Sep 2003

Business intelligence and data warehousing are largely institutionalised. But in spite of its many successes, many organisations are still not fully aware of what it entails and how it could benefit their current and future business. Willie Bezuidenhout, business technologist: information management at Computer Associates Africa, zooms in on data warehousing, providing some valuable background and future assessments - shedding some light on an important data management enabler.

Hailed by some as the solution to the management information dilemma, data warehousing (DW) has probably become one of the most used and abused terms in IT.

One of the key reasons for this is that a lot of companies and individuals still don`t fully understand DW, or even what it represents.

To some, a data warehouse is any collection of summarised data from various sources, structured and optimised for query access using OLAP (online analytical processing) query tools.

To others, it`s simply a database that contains data from more than one source, for the purpose of providing management information.

These definitions can hardly be described as visionary, as they essentially represent support solutions that were in use long before the term DW became part of the IT vocabulary.

Taking one step back

The concept of DW data dates back to at least the mid-1980s. The idea or vision of DW was initially intended to provide an architectural model for the flow of data from operational systems to decision support environments.

In essence, it attempted to address the various problems associated with this flow, and the resultant high-costs.

Based on the concept of "real-life" warehouses, the initial data warehouses were essentially large-scale collection/storage areas for legacy data.

After being collected and stored, data would be distributed to "retail stores" or "data marts" which could then be focused on packaging and presenting selections of data to end-users - meeting specific and specialised needs.

This was a novel vision that provided workable solutions for a number of data management problems.

However, somewhere along the line, this simple but effective vision was lost. DW "gurus" started to emerge at the end of the 1980s, altering the architecture and replacing it with studies on how decision support can be enhanced.

Suddenly the data warehouse had become the miracle cure for the decision support headache.

Assessing the bigger picture

So, where does this put us today and what is left of the original vision?

Although the concept - albeit in various forms - still attracts a lot of attention, many DW projects fail to deliver the benefits expected from them, and many are proving to be excessively expensive to develop and maintain.

It is, therefore, important to look at the bigger picture, making sure you are aware of the long-term benefits, while juggling the costs that will realise them.

The first thing to understand is that the costs associated with DW projects are traditionally quite high. There are a number of reasons for this, but one that stands out is the "collection", "cleaning" and integration of data from different sources - often legacy systems.

This exercise is extremely labour-intensive and indeed time-consuming, but critical to the overall success of a project. The cost of extracting, cleaning and integrating data can represent as much as 60% to 80% of the total cost of a typical data warehousing project.

However, poorly integrated or low quality data eventually delivers poor or worthless management information, eliminating any prospects of return on investment (ROI).

Can you fully realise data warehousing ROI?

Given the fact that the initial costs seem high, how can solution providers justify the viability of a DW project and ensure ROI?

Research authority IDC estimates that business intelligence (BI) solutions, which include DW, offer 112% ROI within a year.

Not too bad, considering the initial investment.

But, not all companies reap this benefit. The main reason for this that companies and their technology partners customise their data warehouses to deliver on specific requirements - limiting flexibility and long-term benefits.

To counter this, emphasis must be placed on the growth of a warehouse as a global resource for unspecified future decision support needs, rather than a solution to specific requirements at a particular time.

Over time, the environment will then grow to offer a permanent and invaluable repository of integrated, enterprise-wide data for management information. This in turn will lead to significantly reduced time and cost to deliver new decision support offering - ROI in the true sense of the word.

The effort to achieve this must not be underestimated, however. Identifying which data is "useful" requires a great deal of expertise and insight. The way in which data is modelled in the warehouse is absolutely critical - a poor data model can render DW obsolete within a few months.

DW can offer enormous benefits to most organisations if they approach it correctly. This in turn will realise a flexible, long-term information delivery environment.

Share

Editorial contacts

Willie Bezuidenhout
Computer Associates Africa
(011) 236 9111
Willie.bezuidenhout@ca.com