Group enterprise data warehousing

Business intelligence (BI) has become the de-facto mechanism for performance management in larger and even medium-sized organisations. If implemented, maintained and managed correctly, the enterprise data warehouse (EDW) is the most suitable place to get the single correct integrated version of the truth across the enterprise.

The EDW is the one place where executives and line managers should be able to directly access the information representing the real truth about the state of the business, consistently, without having it adjusted to reflect a better picture through a zillion spreadsheets along the way to the top.

However, in a network of enterprises, such as in multi-national and multi-company groups, the EDW may just not be enough. If the group is business-savvy, it will realise that by exploiting the synergies between its members and get a consolidated view of clients across these, it can achieve a competitive edge over the individual competitors of the participating companies.

In this case the group would want to improve its performance across the set of participating enterprises, and will need to have a consolidated and integrated view of finances, clients, products, business processes and other prime data entities across the group. Even the efficiencies of supply chains across the group's members can be improved. What is required at this level is cross-enterprise BI, implying the need for some sort of cross-enterprise data warehouse (CEDW).

Cross-enterprise consolidation of financial information is an age-old problem that accountants, auditors and even the taxman are very well aware of. One of the biggest challenges in this field is what the data warehouse designers call conformance. How do you get the financial reports of the participating members to conform to the group accounting structure, when they each have different internal accounting structures? In a multi-national group of identical companies operating in different countries you may still enforce a uniform accounting structure, with difficulty, but with a group of heterogeneous member companies in different lines of business this problem is much more complex.

How to implement and manage cross-enterprise reporting without compromising the privacy and "ownership" of proprietary data remains one of the big challenges for the future of BI. Data ownership in the intra-organisation context is already a dirty word - across companies it becomes a plague. Information custodians have to be made aware that they don't own the data - they are merely its custodian, in exactly the same way as they do not own the organisation's money, manpower or machinery.

But the sharing of information across company boundaries does raise another issue, namely that of privacy. New legislations around the privacy of personal data make it very difficult to share customers' profiles across the members in the group.

So how can the CEDW be implemented to facilitate cross-enterprise BI? There are a number of approaches.

With a service-oriented architecture, the BI competency centres of the participating members can wrap their respective BI capabilities and offer these as services in the architecture. In this way, cross-enterprise scorecards and dashboards can be run by directly addressing the respective internal BI environments of each individual enterprise through the SOA. Thus the data custodian of each participating member is in full control of what information is released to the group.

The main advantage of this approach is that huge masses of data are not replicated throughout the group - only those required for the scorecard or dashboard under review are extracted and communicated to the group level. This results in minimal communication overheads and minimal privacy and security issues.

The big disadvantage though is that mass data exploitation mechanisms cannot be employed optimally. In other words, the dashboards and query tools employed at the group level cannot perform optimised join and union operations across the group. Neither can this approach facilitate drill-down to a detailed level across members. (Single member drill down can provided if that has been made available as a service.)

The other workable approach is to develop the group-level CEDW, much the same as the individual member EDWs are built, but this time using the individual EDWs as source systems. Where there are political issues as to where the CEDW should be maintained, it can even be outsourced to a third party to host, provided they can guarantee the necessary availability, security and privacy. For a large multi-national, it can be even be hosted at one of the biggest member companies, eliminating the need to transport their detail data across the network.

The advantage of this approach is that standard reporting and dashboarding tools can be deployed directly on the CEDW, with full drill-down capabilities. It is a straightforward task to run a dashboard or scorecard on the CEDW, and drill down straight into a troubled member's detailed data.

The disadvantages about this approach are the regular communication of large volumes of detailed data, as well as the political issues around ownership and access, and the legislative aspects around privacy and security.

There will be more developments in this area - the requirements for cross-enterprise BI are just getting more and more by the day. The CEDW is here to stay, and its contents and utilisation will grow. In the highly competitive business world, business advantages such as synergies across the group have to be exploited to achieve that extra edge to get ahead of the competition.