About
Subscribe

Data, data everywhere, but not a drop of value

With the abundance of data immediately available to businesses and individuals on almost any topic, nobody can argue that we have entered the information age. But abundance does not necessarily mean value.
Julian Field
By Julian Field, MD of CenterField Software
Johannesburg, 10 Dec 2003

The information age is both a blessing and a curse. While we may know about every uplifting operation Britney Spears has had, most people and businesses are finding it harder to obtain information that`s relevant to specific tasks and operations at hand.

Information availability is critical to business, but, as with seawater, if you can`t extract the salt and the drudge, you won`t get pure water and will eventually die of thirst.

But this is the IT world and things can never be that simple.

Julian Field, MD, CenterField Software

The problem revolves around quantity as well as quality. We are in a technology-enabled world where everything a person does can be recorded in a database somewhere, whether there is a reason to save this data or not.

Whether it`s Web sites you have visited or the calls you have made to your bank`s call centre, it`s all stored somewhere in case someone wants to use it at some time. And it`s all useless.

That is, it is useless unless there is a way to cut through the volume and find small, relevant, useful bits of information of value to the business in the shortest possible time. The key is the ability to manage data effectively and this is only possible through metadata.

Making data make sense

The common definition of metadata is "data about data", a phrase that can be interpreted in many different ways. Fundamentally, metadata describes what information is, who stored or accessed it, who can view it, and its relationships to other data.

Metadata is not merely a complex abstraction. It has real-world applications we all deal with, such as the ever-pressing question of data privacy. With users demanding more privacy while also insisting on better service, metadata assigns privacy values to data and therefore also the appropriate access rights.

A useful analogy explaining metadata is to imagine a can on a supermarket shelf that has no label. How do you know what is in the can? You can`t, but by adding the label with all its information (the metadata), we know what is in the tin, who made it, when it was made, the ingredients, sell-by date, who should use it and so forth.

But this is the IT world and things can never be that simple. As the amount of information has soared, so has the amount of metadata. Metadata is like interest on a account: it accrues over time. It also grows with use.

The care taken in the design and architectural phases will, like a careful investment, determine its success or failure.

And there are still areas where metadata fails to deliver and its natural limitations are exposed.

All metadata is part of a schema, a definition of data categories and relationships that enable IT systems to make sense of data. Each schema has predefined categories describing the data, any relevant connections to other data and expectations arising from that data.

An oversimplified example: data categorised as salary may be relevant to employee names and we may expect the name to be coded as a staff number, which we will also find on an ID card which has its own number.

Planning is crucial

Metadata works well in most situations where data is limited to certain focus areas and the schemas are fairly simple and predictable (such as an insurance company`s data). When businesses want to expand their metadata to include other companies - for example, all the companies within a large group - problems arise.

It seems logical that one metadata schema can be mapped to another and the two - or as many as required - used together. In reality this is not always possible as each schema is designed for a particular set of data, to be used in a particular way. Each has its own perspective on life, as it were, and the only solution to this conundrum is care in the initial design phase.

Metadata will be the defining element of the information age. The ability to easily understand data and its relationships to other information is a critical aspect to managing our growing mountains of information. The core factor for metadata to fulfil its function effectively is to take care in the planning and design phase to ensure all contingencies are catered for and an accurate, yet easily usable abstraction of the data is gained.

Share