Turning data into knowledge

This week we examine the technologies and associated tools that enable an organisation to implement a successful information delivery solution.

Your enabling technology needs to provide some key integrated 'data management` capabilities.

Last week we focused on the information delivery vision and discussed the information value chain. This week the focus is on the enabling technologies and the tools that make this vision a reality. There are several enabling technologies that make it possible to extract value from the data that you collect on a daily basis.

Referring to the information value chain discussed last week, the second value-added process (labelled "Information") uses data warehousing as the key enabling technology. This process focuses primarily on extracting data from the company`s operational systems, and turning this data into business information. This is then stored for later use.

For this process to succeed, your enabling technology needs to provide some key integrated "data management" capabilities.

So, what is the ideal tool-set that will help you to accomplish your information delivery vision?

In a typical organisation the environment is rather complex. The data resides on multiple hardware platforms (from mainframes to mid-range machines to PCs) and in a variety of data formats.

Firstly, you will require the necessary tools to access this data, no matter where it resides or what format it is in. Once the data has been acquired the next step in the process is to transform it into useful information. This will entail validating, cleaning and integrating it from different sources. The tool-set should have powerful data manipulation characteristics.

Having turned the data into information (or added value to it), the third step is to load and store it in a format that can be used by your business community. Your tool-set should include relational and multidimensional database capabilities as well as the ability to take advantage of multi-processor hardware architectures (parallel processing). Also, you should choose software that features different loading strategies such as "complete data refreshment" and "changed data capture".

In order to automate the steps of data acquisition, transformation and loading, you will need to employ scheduling (native scheduler and the ability to integrate with multiple third-party schedulers) and metadata (data about the data that will be used by IT and business users) functions. Also make sure that you choose a tool-set with an easy-to-use graphical user interface that "integrates" all the above mentioned abilities.

Only once your information has undergone the process of acquisition, transformation and loading is it possible to exploit the final product through a number of different "business intelligence" enabling technologies. Business intelligence is the process (the third step in the information value chain) of turning information into knowledge. You can then provide your business community with a suite of integrated tools that can be used to exploit (add value) to this information.

Some of the key enabling technologies include online analytical processing (OLAP), Web enablement and data mining. OLAP is similar in concept to online transaction processing (OLTP), except that it is information rather than transactions that are being processed.

What tool-set should you be looking for to accomplish this?

An OLAP tool provides the ability for you to view information through multiple dimensions (eg sales by customer and product, sales by region and product, etc). The tool should also offer you the ability to drill down, traffic lighting, hot spotting and the ability to "swap" dimensions at the click of a button.

The Internet/Web provides another more cost-effective vehicle for disseminating applications and information to large numbers of users. You should identify those tool-sets that offer both static (Web publishing) and dynamic Web capabilities.

Data mining is a set of powerful techniques used to scan large volumes of data (eg customer information), spotting trends and patterns, identifying opportunities and pre-empting problems. You should also choose a tool-set that includes a comprehensive set of techniques (visual exploration, clustering, correspondence, neural networks, tree-based modelling, statistical techniques, time series analysis, and more), together with a well-established methodology.

Some of the other tools that you can use to exploit information include querying and reporting, enterprise information systems (EIS), data visualisation and rapid object-oriented application development. Again, it should be emphasized that such a tool-set should be highly integrated.

Next week we will look at the information delivery architecture that makes this all possible.