BI focus has shifted to the back-end

Comedian Carol Leifer once said: "Making love to a woman is like buying real estate: location, location, location." That may be what it`s all about if you love women and property, but it`s all about the back-end in business intelligence (BI), says Julian Field, MD of Centerfield Software.

The BI market has changed and CTOs planning IT strategies need to take note. In the past the emphasis in this market was on front-end tools but now that they have become commoditised and integrated with a variety of back-end systems, primarily through new Web-based technologies, the focus on BI projects is shifting to a number of back-end issues. Correcting these promises better BI system performance, reducing analysis and reporting times; closer alignment of technology and business objectives; and better quality results.

By way of example, IDC estimates that in Asia/Pacific the market is expected to grow by 12% between 2003 and 2004, up from 7.5% between 2002 and 2003. By 2008 it predicts that this market, excluding Japan, will be worth $382 million, with adoption focusing primarily on end-user query, reporting and analysis software tools.

Gartner estimates that companies in 2012 will deal with 30 times more data than they were in 2002. Performance management is also demanding that IT and business executives work more closely to define business processes to be automated, establish business rules for performance measurement and associate metric owners with corporate data for accountability.

Trek to the summit

00Days

:

00Hrs

:

00Min

:

00Sec

This is a problem for companies that deployed data warehouses four or five years ago because the focus then was on the front-end, and the back-end architecture, although right at the time, must now change. The best approach for them is to redesign the entire data warehouse and redeploy it. Not doing so can result in poor performance, poor quality information and not meeting business requirements.

The current short-term correction to slow or unresponsive data warehouses has been to throw more horsepower at it. Even on the extraction, transformation and loading (ETL) side of BI, some local companies are taking up to 12 hours to load data into a BI front-end tool to start analysing it. That`s just poor architecture because the tools, in a properly designed system, can deliver far better performance.

As an example, a large local services organisation bought and deployed a popular ETL software tool five years ago. The business recently upgraded the hardware powering it to an eight-way server to deal with growing data volumes and poor performance. After little performance gain was realised, the company informed the vendor that the five-year-old software tool has been poorly implemented. What they had failed to take into account was that the software was written for its day and deployed on the hardware available at the time, using a maximum of four processors. With little tweaking, the vendor in question had the ETL tool using all eight processors for a massive performance gain. That proves that old implementations must be revisited.

Another aspect of the back-end focus is data quality. Some companies still perform this critical activity manually through the writing of scripts, but the ideal is to automate this process through software. Another issue is that the last time many organisations implemented this, if at all, was five years ago when they introduced their data warehouse. Ensuring consistent, good quality data is an activity that must be constantly revisited. The problem many companies face is that performing the task manually is arduous and the automation software expensive while appearing to provide little immediate return on investment (ROI). Companies are also only likely to complete this task once every two years, or more often for businesses in more dynamic markets, which makes the capital outlay during times of shrinking IT budgets difficult to justify. The opening for vendors in this market is a software rental scheme.

Many local businesses are aware of these trends and are ensuring their BI systems deliver on the necessary business requirements, as evidenced by the current plethora of tenders focusing primarily on them.