Five years on: Governance still biggest hurdle to BI

Even with governance more important than ever before, it’s not an easy task to ensure good, clean, quality data is available for accurate BI reporting.
Read time 4min 10sec
Chris Pallikarides
Chris Pallikarides

Almost half a decade ago, businesses realised that effective data governance was key to a solid business intelligence (BI) strategy. However, today, governance is still the biggest hurdle to BI.

KID director Mervyn Mooi noted in 2015 that BI projects built on ungoverned, unqualified data/information and undermined by shadow BI would deliver skewed and inaccurate information.

In a study done by Forbes in 2016, it warned that data governance required a healthy balance between consistency and flexibility.

In the years that followed, we saw the emergence of more automation, artificial intelligence (AI) and dedicated data stewards – ideally, to rectify this.

However, today, while organisations are moving at the speed of business and data is being created at a rate of 2.5 quintillion bytes per day, it is still not an easy task to ensure good, clean, quality data is available for accurate BI reporting.

The mandate

With compliance being a requisite in almost every industry, companies face a massive whip should they not comply with certain reporting standards.

Now, as it was then, governance is more important than ever before. Having a solid governance framework in place is vital to ensure the data being used to make critical and non-critical decisions is as accurate as it can be.

How can a business make a decision if it has no trust in its data/information, or where the data is coming from?

In the world we live in, where mission-critical or even life-threatening decisions need to be made quickly, data/information needs to be available immediately − and that data needs to be trustworthy and correct.

With self-service reporting now available to almost everyone, being able to report information and present data accurately is critical. Having a strong governance framework in place ensures the data used can be trusted and that all the relevant controls, checks, standards and legalities (compliance) are adhered to.

The method

Modern day governance tools enable organisations to map and link all data/information and business and technical process artefacts or assets (metadata) to each other, to the respective systems, role-players or stakeholders and also to compliance/regulatory rules and clauses as they relate to each other.

Using in-built AI or manual interrogation or discovery methods, this repository of mappings immediately enables true lineage, landscape oversight and proving of compliance.

How can a business make a decision if it has no trust in its data/information, or where the data is coming from?

It also surfaces opportunities for efficiencies; ie, overlaps in artefacts, manual lineage and gaps, change tracking or role actions.

The governance model is premised upon oversight and control over data management (including data quality, verification, security) processes, roles and compliance to external regulation or internal standards.

Governance guidelines are available from many sources on the web and specifically in tool vendor documentation.

To make governance easier, it is recommended that organisations make relevant such guidelines within a site-specific framework of relevant standards that are realistically applicable or pragmatic; ie, articulated.

Data governance sets a blueprint of controls for the proper management of data assets across the organisation.

The challenges

Even with governance processes and tools in place, effective governance can be challenging because data changes all the time. Customers may move and organisations may change their reporting, branches or processes.

Therefore, linking and change tracking is crucial in ensuring data integrity and accurate current and historic reporting.

Matching and merging of historic data to ensure design and storage conventions are aligned and all data is accurate according to set rules and standards may require up to a year of careful design and architecture to integrate data from various departments and sources, in order to feed the BI system.

Another challenge is the existence of ‘rebel’ or shadow data systems, in which departments start working in silos, creating their own spreadsheets, duplicating data and processes, and not inputting all the data back into the central architecture. This obviously results in huge and unnecessary costs.

Even data as apparently straightforward as a customer’s ID number may be incorrect – with digits transposed, coded differently or missing. Data stewards are still required to carry out certain manual verifications to ensure the data is correct and remains so.

The proposition

Stronger governance helps ensure not only consistent but also reliable and optimised results.

It all needs to come through the central architecture or common model regulated by framework standards.

In this way, the entire ecosystem can be governed effectively and data/information can be delivered efficiently and in a trustworthy manner, also making management easier and more cost-effective.

Chris Pallikarides

General manager, ITBusiness, a company in the KID Group.

Despite a school aptitude test predicting he would be best suited to the world of IT, Pallikarides studied his LLB at UNISA and started his articles before finally embarking on his career in IT.

He began his IT career 15 years ago at the IT ‘coalface’ – working at the IT helpdesk of Lancet Laboratories. He then focused on sales and business development, working at Med-e-Mass and a technology reseller.

As business development manager in the IBM Software Business Unit at AxizWorkgroup, Pallikarides gained valuable experience helping clients to design solutions that rested heavily on security and data management; at the same time honing his business development, sales management and strategic business skills.

Having gained experienced in solution development across diverse fields, including healthcare, banking, finance and retail, he was appointed as GM of ITBusiness in 2019.

Tasked with growing the firm into the next-generation solutions space, Pallikarides draws on his legal background and expertise in data and solution design to take ITBusiness into its next phase of growth. 

See also