BI value depends on data quality

There is a fundamental shortcoming in most business intelligence (BI) tools in that they cannot support the full range of BI functionality in a single architecture - leading to excessive costs, delays and user dissatisfaction.

Using one tool to integrate all styles of BI results in lower total cost of ownership (TCO), ease of integration, reduced need for training, improved load balancing and tighter security. BI applications have evolved dramatically over the last decade as companies` appetite for information has grown, along with their demand for more ways to report on and analyse data. The expansion of data warehousing and the widespread adoption of enterprise applications, such as enterprise resource planning (ERP) and customer relationship management (CRM) have fuelled the demand for BI reporting and analysis applications.

BI technology vendors have traditionally built niche software to implement each new pattern of application that companies invented. These resulted in software products centred exclusively on one style of BI - each representing a certain characteristic end-user function.

1. Enterprise reporting: Broadly deployed report formats for operational reporting and scorecards/dashboards targeted at information consumers and executives. These are destined for broad distribution to many people.

2. Cube analysis: OLAP slice-and-dice analysis of limited data sets, targeted at managers and others who need a safe and simple environment for basic data exploration in a limited range of data.

3. Ad hoc query and analysis: Full investigative query into all data, as well as automated slice and dice OLAP analysis of the entire database - down to the transaction level of detail if necessary. This is targeted at information explorers and power users.

4. Statistical analysis and data mining: Full mathematical, financial and statistical treatment of data for purposes of correlation analysis, trend analysis, financial analysis and projections. Targeted at professional information analysts.

5. Alerting and report delivery: Proactive report delivery and alerting to very large populations based on schedules or event triggers in the database. Targeted at very large user populations of information consumers, internal and external to the company.

In forward-thinking companies, departmental BI is being replaced by one industrial-strength BI platform that can deliver all five styles of BI.

Charl Barnard, GM, Knowledge Integration Dynamics

The end-result of departmental BI tool limitations and departmental BI purchasing has been the proliferation of islands of BI that we see today. But the time has come for departmental aspirations to give way to company requirements, and in forward-thinking companies, departmental BI is being replaced by one industrial-strength BI platform that can deliver all five styles of BI. And there are good reasons for this:

* Enterprise-level BI applications access more data and support more users, while departmental BI lacks user and data scalability.

* Departmental BI results in inconsistent versions of the truth that are propagated across the company - multiple islands of BI result in many, incoherent metadata repositories.

* Users are increasingly dissatisfied about being forced to use multiple BI tools.

* Enterprise-level BI applications require a richer user experience encompassing multiple styles of business intelligence - single-purpose BI tools cannot mix and match styles of BI.

* IT organisations cannot afford the excessive cost of managing multiple BI technologies - disparate BI technologies for multiple BI applications are burdensome and require much training.

Companies embarking on BI initiatives, or those wanting to enhance the value they receive from their existing BI tools, must ensure the necessary systems integrate smoothly with each other to enable the rapid movement of data between and through them, and that the data exchanged between them is of the right quality. Poor-quality data results in ill-informed business decisions, poor CRM and regulatory non-compliance.

Although companies are beginning to understand the need to profile their data, ensure high data quality and enable integration, most are doing these as three separate projects. The business can achieve more value if companies approach profiling, quality and integration (PQI) as a single project.

A key element of data cleansing, data profiling enables companies to understand their data, and the processes of how it is compiled and acquired. It is an analytical process, which creates metadata from existing content and provides insight into the business logic behind data construction. The key to data profiling is that it does not change the data in a company`s databases, but discovers what data actually resides in each column and row, and creates a map for the business to follow.

The value of BI systems relies heavily on the quality of a company`s data, which is becoming more and more complex, with companies collecting and retaining mounds of data, from customer, product, warehouse and financial data, to partner, vendor, supplier and employee data. But more often than not, this data is of poor quality, and flawed data corrupts and undermines almost every IT and BI project. Corporate data is a key strategic asset and the goal of data quality management is to provide the foundation for creating information and knowledge.

The benefits of good-quality data include cost savings, improved decision-making, better customer service, improved chances of marketing success, new opportunities to cross- and up-sell, improved forecasting accuracy, smoother operations and supply chain management, and regulatory compliance.

The information that runs a business resides in different formats across countless systems, from ERP to CRM, financial management, and customised in-house systems. If these systems cannot communicate with each other or share data, the value they hold for the business is limited, and they will be incapable of catering to the individual information needs of users, regardless of how well profiled the data is, or of its quality.

Profiling, quality and integration make up the corners of the PQI triangle. If one of them is removed, that triangle no longer exists, and the effectiveness of PQI in enhancing a company`s BI initiatives is reduced.

Applying PQI as a collaborative effort, and with an integrated BI platform, enables a company to successfully turn its data into an asset that can be leveraged.

* Charl Barnard is business intelligence GM at Knowledge Integration Dynamics.