Data quality strategy

Implementing data quality management and its associated controls in an organisation is a very complex task. This is due to many reasons, among these is the fact that data span organisational boundaries and move backward and forward across these boundaries, often in uncontrolled ways or through undocumented channels.

In most organisations there are also no clear distinctions of ownership and custodianship of data, which in turn blurs the responsibilities related to data quality.

You may ask: why would a CEO be concerned with the two terms that make up the term data quality? In his eyes maybe "data" are the basic building locks of information that are manipulated by the organisation`s computer systems, and "quality" is an indication of the workforce`s dedication to their work. But it is not that simple...

Bad data quality can cost an organisation dearly. It can lead to fraud cases and records of some of the worst decisions in history reported on national TV and headlined in the daily press. In less extreme cases, statistics show bad data can easily cost an organisation as much as 10% of its revenue in wasted effort trying to deal with, or work around, bad quality data. A decision based on bad quality data is nine times out of 10 a bad decision. The amount of effort it takes to convince a board of directors to reverse their opinion after distrusting a statutory report`s data quality can become a management nightmare.

In some industries regulatory data management legislation is forcing organisations to even report on data quality measures, in conjunction with auditability and risk analysis.

To achieve our business objectives, we need appropriate, complete and accurate data in the right context and format at the correct time and place for the right purpose. In other words, we need high quality data. At the end of the day, the objectives of data quality management are in line with our mainline business objectives: to decrease costs and increase profits.

The chief financial officer is interested in "where is the money?" To him, all aspects - including data quality - of financial data and financial reporting is important. Similarly, to the head of marketing in-depth and accurate knowledge of the customers and their behaviour patterns are crucial. Operation managers are interested in product data, staff productivity, sales force effectiveness etc, and in the trenches the line managers have to ensure that departmental systems are used correctly. Do any of these have a direct responsibility for enterprise-wide data quality? Definitely not. These roles are too specific and too narrow-focused to be able to worry about the big picture of enterprise data quality. Even the newly appointed compliance officer only has to focus on regulatory reporting issues.

Enter the chief information officer, or CIO, whose ultimate responsibility is the management and control of the data and information resource of the organisation. (The CIO should not be concerned with technical details about servers, networks or software licences and usage performance statistics. That is the domain of the chief technical officer, or CTO. If the CIO is dealing with these technical aspects, his job title is simply wrong. It should be corrected.)

Just as the CFO deals with the financial resources of the organisation, so the CIO deals with the data and information resource of the organisation. The CFO is not responsible for every channel through which money enters or leaves the organisation, but he is ultimately responsible to keep accurate record of all the money he becomes the custodian of. Similarly, the CIO is not responsible for all data flows into and out of the organisation, but he is responsible for the quality of all the data that he inherently becomes the custodian of. Just as nobody ever "owns" the money, nobody ever "owns" the data - they are merely the managers and custodians thereof.

Ultimately, the CIO is responsible for data quality. Especially due to the cross-boundary flows of information, its quality has to be managed from the executive level. Line management does not have enough cross-functional clout to enforce enterprise-wide data quality controls.

Managing data quality is much more complex than managing product quality or even service quality. In addition to often badly defined cross-boundary flows, non-ownership and various other known problems, bad data quality is like a virus - you never know where it will turn up next or what damage it will cause.

In order to properly manage data quality across functional boundaries and through departmental silos, business buy-in is required across the board to make this happen. Data quality management takes time and resources - from the CIO`s office as well as from all the various line functions. This has to be motivated and accounted for.

Other complications include:

* Existing perceptions: if people have made up their mind that the quality of the data is bad, it is very hard to have them make a 180o turn away from that perception - it takes 100 positives to correct a single negative.

* Legacy data: Old data, originating from old archaic systems, is very hard to improve, especially if the old systems are not top priority any more.

* Informal data (in the form of Excel, Access, and other data) on users` PCs, are a data quality management nightmare. Each of these data "sources" are a potential data quality threat.

* The "accidental data warehouse", which came into being to address a few short-term requirements way back when, but which has never been properly architected or engineered is another serious data quality threat. The chances that the organisation is going to throw good money at it are almost zero - especially if it hasn`t delivered anything of value in recent times.

* Yesterday`s requirements - where the business has new and urgent requirements which always have priority over data quality management - are the death nails in the data quality coffin. If this situation persists, it implies data management is not being driven and managed from high enough management levels.

* Existing power-play, politics, private agendas, uncooperative role-players and domains of control will always counter enterprise-wide initiatives.

Data quality management has its own disciplines, methodologies and processes. These have to be employed enterprise-wide to have any effect. Many governances and controls are required on so many levels. The feedback cycles from the information consumers (typically management and board and management structures) to the data producers (typically line functions) are often so far removed from each other organisationally, that these feedback cycles have to driven through top management. The only way to get it all done is through an enterprise-wide accepted and top-down driven data quality strategy, under the constant drive of the CIO.

Data is viewed and managed as a primary corporate resource. The quality of the data directly affects the bottom line - for that reason, data quality management should be driven by the CIO from the executive level.

Data quality management is an end-to-end process, from transactional source systems, through the data flows and data storage in the data warehouse, to end-user, management and executive reporting and analysis. For that reason, an enterprise data quality strategy is required, which must be used to drive the data quality management approach and its implementation throughout the organisation. The key aspects to implementing such a data quality strategy are teamwork between business and IT to form an effective partnership and the avoidance of politics through executive-level drive and effective communication.