Johannesburg, 25 Nov 2019
Searching for solutions to organise and instantly understand data lineage can be daunting. Ensure you choose a data lineage solution that offers deep and truly end-to-end data lineage you can trust. Below are five criteria that differentiate one solution provider from another and will help you choose the right solution.
How deep is the data lineage provided by your solution?
You need insight into where your data originated and how it was constructed. From there, your solution must offer visibility into all the applications that influenced every key data element. Finally, demand that your solution offers analytics on your lineage. Generating deep insights make your organisation act and react smarter. Deep insight is not possible without deep technical lineage. Your chosen solution must tackle the lowest level technical data lineage possible.
Does the solution provider offer a wide breadth of supported inputs?
Can your solution read code out of the box, work with COBOL, SAS, mainframe and big data environments? Can your solution parse all the different flavours of SQL databases? Your data governance solution should be able to expose data transformation logic at the lowest level. Your data governance solution should simplify, not make things more complicated. This means you should be able to view the lineage at different levels of aggregations – from business level to data element level, including any and all data transformations.
Your organisation is complex, so your solution needs to address the technologies you rely on. Seventy percent of companies still rely on mainframe data to run their business. Why would your lineage solution ignore such a large portion of your data? Make sure your lineage provider covers all the technologies on which you rely.
Does the solution provider have a rich understanding and practical experience building end-to-end data lineage in organisations across the globe?
Your data governance/lineage solution provider must have practical applied experience in implementing such solutions at many organisations similar to yours. They should be able to predict data lineage challenges before they arise and deliver end-to-end data traceability more quickly and efficiently than anyone else.
Can the solution provider quickly build custom data lineage extractors?
Data lineage isn’t easy and no two occurrences are alike. If a special lineage capability is needed, your solution experts should build it. And they should be able to build such a custom lineage extractor quickly. In any reasonably complex environment, there will likely be technologies for which there is no ready-to-deploy lineage extractor available in the market. This will create a gap in your lineage. Therefore, choosing a solution provider that has the depth and breadth of experience – allowing them to build custom lineage extractors quickly – is critical to your capability to provide end-to-end lineage.
Does your solution provider just “scan” the metadata superficially or “extract” it deeply?
Relying on solutions that simply scan data relationships leaves you vulnerable and unprotected. Scanners do not identify problems that lurk beneath the surface the way extractors do. Imagine using a GPS system that only provided information about cities and highways, without ever reporting car accidents or alternate routes to your destinations? Or it showed you only part of the way to get from point A to B.
Prefer an infographic?
This infographic contains key points from the above article. Download it now for ease of reference!
Share