While metadata as a buzzword has come and gone, organisations embarking, for example, on a software development, integration or database design project will feel the need for it.
Most large local organisations are only now beginning to delve into data profiling with the intent of developing a metadata repository or master data repository, but few of them have an idea how to achieve it. These businesses have between 3 000 and 5 000 employees.
One of the primary issues these companies state for not having performed this in the past is the difficulty in identifying return on investment (ROI). Since developing metadata through data profiling is most closely associated with cataloguing a library, it is not a task often performed, it is costly and there is little visible ROI. The ROI is discovered only if the catalogue is properly maintained, and a long time from the date of institution.
But there are benefits: Investment management firm T Rowe Price in the US was developing an integration competency centre (ICC). Before establishing the ICC, the firm took three months to ensure it had the best practices, processes and infrastructure in place. It also created naming conventions and standards and set up a metadata repository to capture appropriate information. T Rowe Price`s ICC is currently handling 15 projects for the business and has reduced maintenance requirement and costs; improved consistency of development; improved data quality; and is aware of how data is being used in the business.
The metadata issue was most clearly highlighted with resolving the Y2K problem. The difference between virtually no Y2K problem and running the IT shop 24x7 to find and resolve Y2K issues, even between departments within a single business, was largely a matter of metadata. Those with a history of good metadata management were the fortunate ones.
Those with a history of good metadata management were the fortunate ones.
Julian Field, MD, Centerfield Software
Another example of the state of data profiling and the metadata requirement is a demonstration I recently conducted at a large local business. I ran the data through a profiling tool and it told us what the data looked like, giving us a good idea of the fields that were related to one another, what the columns contained and the uniqueness of fields. As we expected, the customer number was unique but the South African identity number was not, which meant there were more customers listed on the balance sheet than actually existed. In a case like that, the profiling and resultant metadata will lead to a data matching and cleansing exercise that will yield direct results for customer relationship management and other marketing and customer retention and growth projects.
What these businesses are only grasping is that metadata is important for many reasons:
* Overcoming the effect of staff turnover
* Enabling data reuse and updating
* Documenting data sources
* Facilitating data transfer
* Providing information for catalogues
* Providing flexibility for searching
* Prevents expensive disasters
* Cuts costs
* Reduces workload
* Prudent stewardship for good governance
As a parallel, nutritional information is contained on food packaging to ensure consumers can make an informed decision about the products they eat. Without it they might never know what they are eating.
In the past consultants performed data profiling and metadata compilation, and the larger the business and more complex its systems, the more of them were required to do so. Today software is available that automates the entire process that reduces costs and reduces the time required to complete data profiling and metadata generation. Essentially this market is not a money spinner for vendors, which have put themselves in direct competition with the consultants; however, it is a necessary prerequisite for completing the many other IT projects that do offer a visible and obvious ROI.
Share