Who`re you gonna call?

By Martin Rennhackkamp, director at Prescient Business Technologies

In most business intelligence programs, we have a requirement for a lot of data and metadata - we need data and metadata from each operational source system and we provide data and metadata about the data warehouse and the downstream data marts. The data warehouse metadata may again be required in other downstream initiatives, such as for analytical applications. The big question is - when we want such data and metadata, who`re we gonna call?

In many organisations we are now seeing people filling the positions of CIO and data architect - which is a great step in the right direction. However, are the persons filling those positions doing what is required from the role? And most importantly, do they have skills and experience necessary to do the job?

The data architect should be the information booth of the organisation when it comes to all things data-related - be it metadata, data in operational source systems or data as remodelled and enriched and presented as information from the data warehouse to the business users through reports and analytical applications.

In data terms, the data architect should have the data model of each and every database and application in the organisation, and should also have an entity life history of each and every data entity that plays a role in the organisation.

Just a week ago, I requested a data model of a source system that was to be loaded into the data warehouse... no, was the response, the data architect would not have the model; you should ask the DBA of the source system for a database definition script... Now, firstly, the DBA of the source system should be busy making backups and tuning his database to fly at mach 3, and should not be required to provide data models. Secondly, then what does the data architect have, and what does he actually do? Thirdly and most horrifying, does that mean there is no enterprise data model? In which case there is probably no metadata either... Scary enough to call the Ghostbusters, for sure.

The company accountant would be able to inform you of each bank account the organisation has, what its opening balance, transaction movements and closing balance for each month was. In addition, that accountant will be able to show you the exact path money flows through the organisation - from where it enters at a till in a store or as payment of an invoice, through the bank accounts and eventually out as salary, invoice payment or a share dividend.

Similarly, the products manager will be able to describe and provide documentation of each and every product, where it is sourced from, where it is used or sold and where it is dispatched from. The HR manager would tell you the same about each and every employee - when they joined, what their promotion path was, where they are now, how much leave they have had, what courses they have attended, etc.

Likewise, the data architect should be able to show you the lifecycle of each and every data item through the organisation`s systems - where the data items are created, what characteristics they have, where they are used, where they are archived or disposed of, etc. This is highly necessary because, doesn`t the data describe the flow of the money, the products, the people and more throughout the organisation?

In this era where managing the data about the business is rapidly becoming the business, companies must take control and manage their data resources properly.

The position of data architect should be staffed from among the best and brightest of the IS organisation - the person should possess a broad knowledge of technology and system development in order to comprehend the full lifecycle of each and every data item. The project manager should also know the technologies and methodologies used on each project and in each system, in order to provide guidance and sound judgment on data management requirements. The data architect must have a sound knowledge of and experience with the systems components making up the entire business systems architecture.

Because of its unique characteristics, training and experience on data warehouse architectures and infrastructures are necessary. Experience with conventional systems design, implementation and systems architectures is also necessary.

Strong conventional data modelling skills are beneficial, as the data modeller often has to be able to read and understand existing data models, but the data modeller must also be able to break away from traditional relational OLTP and E/R design practices and embrace the new data warehouse design techniques as well. Ideally this person should have been trained in a data warehouse design methodology, and should have had some experience working on other BI projects under an expert data modeller. It is also helpful if this person has a solid understanding of the business rules, as he or she often participates in the business requirements definition activities in a secondary role.

The data architect should have strong communication skills, as well as a strong affiliation with and the respect of the business community. The data architect should have a quick grasp and understanding of the business, the corresponding data models, and should be able to see things abstractly, for example to derive common data models across business functions and to represent related business concepts generically. This person must also be very meticulous, as the organisation`s data models must be exact and complete - attention to detail is a very useful characteristic.