Overcoming conformance challenges

One of business intelligence's many challenges is the ability to combine and integrate data into information that makes sense across the entire enterprise. For a small enterprise this is often not a problem, because the entire business can be run off a single system, but for large enterprises, the challenge is combining information from many different sources, where the same kind of information is often represented in radically different ways.

Any corporation that has been through a merger or acquisition would also immediately appreciate the predicament of having normal transactions being kept in different systems for a set of overlapping or non-overlapping products and clients.

Looking at a business matrix, it is intuitive to see that the columns in the matrix represent the so-called 'measurement context', and in a single large organisation, the components of the context, called dimensions, will overlap from one business process to the next. For the business processes to be analysed consistently across the entire enterprise, these dimensions have to be consistent in meaning and content, no matter where it appears. As we refer to 'meaning' it is important to understand how easy it is for people to misunderstand each other.

This consistency is therefore something that needs to be negotiated and agreed to by all relevant parties in the business and not by IT per se. Ownership and contribution by businesses are key elements that have to be addressed. Once executive buy-in is achieved, a series of meetings between the various stakeholders, with a clear instruction from executive level on the expected outcome, is often needed to get to a business-wide accepted definition of a conformed dimension.

In essence, conformance is as much, if not more, a business issue, as it is a technical issue. In the industry some refer to the larger business problem as 'master data management'. If proper business-driven master data management is successfully done in an enterprise, a lot of the groundwork that is needed to produce conformed dimensions in the BI realm is already achieved.

Once the definition of the conformance has been achieved, the modelling can commence. For example:

* Facts (measurements): A month-end account balance has the same meaning regardless of the type of account. It also has the same meaning to everyone in the business, regardless of whether they work in the marketing, operations or finance department.
* Dimensions (descriptors): Account number has the same meaning regardless of the type of account.

This all boils down to the dimensional business matrix, which should be the basis for an integrated view of any enterprise.

Ownership and contribution by businesses are key elements that have to be addressed.

Cor Winckler is technical director at PBT Group.

The conformed view of the enterprise's data and business processes applies to the enterprise as a whole. As such, it will always be a subset of the entire set of information available at the lowest level of detail. An easy banking example to illustrate this is the case of different bank account types. A cheque account will have some unique attributes that do not apply to a home-loan account and vice versa.

Because these attributes are still important and valid for the people analysing data in the particular business area that generated the data, it would be short-sighted to not bring them into the larger data warehouse.

This challenge is known in the Kimball school of thought as a heterogeneous dimension, and there are known technical techniques for handling the association of the non-conforming information with the conformed business matrix.

Conformance, and the business challenges around it, fit very well into the dimensional modelling paradigm, made popular by the Kimball Group. The biggest business value brought by conformance, however, is the ability to integrate information from various silos in the organisation into a single conformed information bus. This integration is invaluable in solving typical problems like the ever-elusive 'single view of the client', 'one version of the truth', or the ability to do basil risk reporting for the entire enterprise.

Finally, all this integration is possible without discarding the rich base of information available close to the proverbial coalface. This makes it possible to do the entire spectrum of reporting and analysis from detailed departmental requirements, all the way up to board-level summarised information, all from the same integrated information bus - business value in every right.

* Cor Winckler is technical director at PBT Group.