Data architecture determines information effectiveness

In every business, whether local, foreign or a broadly dispersed global corporation, decisions made at board, executive and even line management level must be based on trusted information.

In many cases, this information has been extracted from internal operational sources, filtered through business intelligence (BI) systems, which can include data warehouses, analysis tools and reporting services, before being presented to the decision maker.

These BI systems are responsible for churning out data and analyses thereof to help management identify trends and patterns in the market that can be used to make forecasts and decisions affecting the performance of the company. The trustworthiness of the information provided is obviously a critical factor in the reliability of these decisions and forecasts.

"If every organisation had one database in which all its data was held, BI would be a breeze because all the information needed would be readily available and probably in a standard format," says Mervyn Mooi, technical director of Knowledge Integration Dynamics (KID). "Unfortunately, reality is somewhat different and challenging, as companies are dispersed, data resides on multiple systems and in different formats and the information needs of various people can vary."

To effectively and efficiently overcome this "chaos", companies have had to implement a BI solution that extracts data from these diverse systems, combines and cleans it to make sure there is no redundancy before inserting the new information into a data warehouse. From the warehouse, analyses can be run and decision-enhancing information extracted. However, this is usually a long, complex and costly process that needs constant maintenance and support.

"To alleviate this burden, it is best to formulate/standardise the company`s information foundations and design an efficient data architecture," adds Mooi. "The concept is simple: a uniform and consistent data architecture applied across the organisation will ensure that information for decision-making is easier to collate and analyse, resulting in faster BI processes which can also be applied in near-time."

With a consistent data architecture, information can more easily be extracted from diverse sources. The data still needs to be checked for accuracy and redundancy, but the difficulty of converting everything to a common format before cleansing and adding it to a data warehouse can be avoided for the most part.

"What makes implementing a consistent data architecture even more valuable to companies is that, in certain instances, the business can forego a central operational data store (duplication of source data)," adds Mooi. "Specific summaries of the current data, such as key performance indicators (KPI), can be made available directly from the source systems with a little effort and because this information is in summarised format, it will not clog the network bandwidth and can therefore be delivered in almost real-time at any time of the day."

Summaries of current data may be of use for executives who only need a quick, high-level view of the status of the company, but most users will need drill down facilities if they are to make reasonable decisions. Each information user has unique data needs. With the summarised approach, administrators do not need to create reports and queries for every user`s needs, but can simply provide the same summarised information to everybody, and individuals can drill down to the specifics they need using a report/data link strategy.

Modern applications have taken their lead from the Internet and can offer dynamic links, similar to hyperlinks in Web pages that allow users to build drill paths from summarised information into the originating data sources or underlying data warehouse if the appropriate analytical applications are in place. This will still keep network traffic down and prevent the BI system from adversely affecting operational systems.

"As mentioned, to achieve this Nirvana, a consistent, standard data architecture (models, structure, format and processes) needs to be in place throughout a distributed environment," Mooi explains. "And this is the tricky bit. Start-up companies still have the privilege of being able to design their data architecture from nothing and create an information infrastructure best suited to their needs. Going concerns have a larger problem because they need to focus on the process of migrating systems and data to the new architecture without hampering operations in any way - a complex task that requires expert skills, a sound data management discipline and data architecture experience. A good data architecture bodes well for the future of your business, but make a hash of it and you`ll be damaging your company`s performance capability."