Big data best practices

Perceived high costs, complexity and the lack of a big data game-plan hamper adoption in many local businesses.

Read time 3min 30sec

Big data as a buzzword gets thrown around a great deal these days. Experts talk about zettabytes of data and the potential goldmines of information residing in the wave of unstructured data circulating in social media, multimedia, electronic communications and more.

As a result, every business is aware of big data, but not all of them are using it yet. In SA, big data analytics adoption is lagging for a number of reasons, not least of them the cost of big data solutions. In addition, enterprises are concerned about the complexity of implementing and managing big data solutions, and the potential disruptions these programmes could cause to daily operations.

It is important to note that all business decision-makers have been using a form of big data analytics for years, whether they knew it or not. Traditional business decision-making has always been based on a combination of structured, tabular reports and a certain amount of unstructured data - be that a phone call to consult a colleague or a number of documents or graphs - and the analytics took place at the discretion of the decision-maker. What has changed is the data has become digital; it has grown exponentially in volume and variety, and now analytics is performed within an automated system. To benefit from the new generation of advanced big data analytics, there are a number of key points enterprises should keep in mind:

Start with a standards-based approach: To benefit from the almost unlimited potential of big data analytics, enterprises must adopt an architected and standards-based approach for data/information management implementation. This includes business requirements-driven integration, data and process modelling, quality and reporting, to name a few competencies.

Proof of concept unlocks big value: Key to success is to start with a proof of concept (or pilot project) in a department or business subject area that has the most business "punch" or is of the most importance to the company. In a medical aid company, for example, the claims department or business might be the biggest cost centre and with the most focus. The proof of concept or pilot for this first subject area should not be a throwaway effort, but rather a solution that can later be quickly productionised, with relevant adjustments, and re-used as a template (or footprint) for programmes across the enterprise.

Get the data, questions and outputs right: Enterprises should also ensure they focus on only the most relevant data and know what outputs they want from it. They would have to carefully select the data/information for analytics that would give the organisation the most value for the effort.

Furthermore, the metrics and reports the company generates and measures itself by, must also be carefully selected and adapted to specific business purposes. And, of course, the quality and trustworthiness of sourced data/information must be ensured before analytical models and reports are applied to it.

Business decision-makers have been using a form of big data analytics for years, whether they knew it or not.

Get the right tools: In many cases, enterprises do not know how to apply the right tools and methodologies to achieve this. Vendors are moving to help them by bringing to market templated solutions that are becoming more flexible in what they offer, so allowing companies to cherry-pick the functionality, metrics and features they need. Alternatively, companies can have custom solutions developed.

It's a programme, not a project: While proofs of concept typically show immediate benefits, it is important for companies to realise the proof of concept is not the end of the journey - it is just the beginning.

Implementing the solution across the enterprise requires strategic planning, adoption of a common architected approach (eg, to eliminate data siloes and wasted/overlapping resources), and effective change management and collaboration initiatives. These will help the company overcome internal politics and potential resistance, and ensure the programme delivers enterprise-wide benefits.

Mervyn Mooi
Director at Knowledge Integration Dynamics.

Mervyn Mooi is a director of Knowledge Integration Dynamics (KID), and also a key resource within the company's information management, data warehousing and business intelligence teams. He has been in the IT industry for 36 years, beginning his career as an operator at the CICS bureau in Johannesburg in the early 1980s. Thereafter, he was appointed as a programmer at state-owned oil exploration and production company SOEKOR. In 1986, Mooi joined Anglo American's head office IT department where he remained for almost 12 years. Here he progressed to become a senior programmer, analyst, database administrator and technical support specialist. After completing his degree in informatics, he then left to join Software Futures, where he worked as a senior consultant for 18 months in the data warehousing and business intelligence arena. Mooi joined KID in 1999 as a data warehouse and business intelligence specialist. Mooi's experience in ICT disciplines includes operations, business and systems analysis, application development, database administration, data governance/management, data architecture/modelling, production application and systems software support, data warehousing and business intelligence. He now focuses on enterprise information management, information governance and cloud solutions.

Have your say
a few seconds ago
Be the first to comment