Is your data in a lake or a swamp?
Enterprises need to be able to process structured and unstructured data in order to make valid business decisions. But, all too often, not all of the data is accessible.
Companies have been collecting and storing data for years. However, the realisation is growing that they may not necessarily be able to analyse all of that data. Extracting and analysing data requires businesses to have a big data platform that includes a mechanism to gather data from various sources, collating it in a single location, so that it can be analysed. Only then can the data yield the business insights that create real value.
When it comes to structured data, the technology around retrieving and looking at it is mature. But the explosion of unstructured data has caused a dilemma for companies as they need to base their business decisions on analysis of both structured and unstructured data.
Gary de Menezes, Managing Director of Micro Focus South Africa, cites the example of an executive who wants to insure an expensive sports car. “He does the necessary paperwork, providing details around his salary, previous accident record, etc. Based on that data, he would be deemed to be low risk. However, if the decision-making criteria were to include data from the executive’s social media, which depicts him out drinking with friends and participating in track race days, that unstructured data might give a very different profile of the customer.”
The explosion of unstructured data and how companies can make use of it has been a dilemma for the past 10 years. It’s only natural that companies should look at how they’ve managed structured data in the past, and try to create a similar solution for their unstructured data. De Menezes says: “Today’s businesses are all able to collect, store and keep their data. The goal has been to ensure you have as much data as possible and keep it for as long as possible.”
This led to vendors introducing proprietary storage solutions for dealing with big data in companies. He explains: “All of the big vendors have their own solutions that basically create big pools of data that we call data lakes. Adding to the challenge is that each company has multiple lakes, which can be on-premises, in the cloud or somewhere in between. While these lakes are getting bigger and bigger, they’re also increasingly isolated from each other.”
Adding to the complexity and cost for companies is that legislation changes require data to be stored for seven years, which has a cost implication that is exacerbated when data lakes can only be processed in isolation. “Companies effectively need to increase their data storage costs and capabilities by 40% in order to comply with legislation. If the company’s big data is kept in separate proprietary data lakes, this adds to that cost.”
It’s clear, enterprises face a major challenge around how to actually get to all of their data and use it meaningfully so that they can monetise it. Kobus Robinson, PreSales Director for South Africa at Micro Focus, says there are some excellent examples of this being done successfully by companies in the retail space. “Retailers are using data analytics based on data collected from applications on customers’ smartphones to push special offers to them as they walk past sensors in their shops. They know the customer’s preferences and buying history and tailor the messages accordingly.”
He says: “Companies across all sectors, from finance to insurance to telecommunications, are all striving towards a single view of the customer, but are struggling to achieve this.”
Big data has become the new elephant in the boardroom because of the cost of managing it as well as because of concerns that it might be adding no real value to the business. This is leading to a new addition to the C-suite – a chief data officer – whose sole responsibility is making sense out of all of the business’s data.
The CDO must take the business’s investment in big data and see how it can be used to add value. This raises the question: when is big data valuable? Big data on its own is absolutely useless to an organisation without other elements incorporated, such as the ability to gain proper insights, perform number-crunching and self-learning. “Value is derived when big data offers actionable insights. However, all too often,” says De Menezes, “the issue is that there’s simply too much data. This forces the business to create more data lakes in an effort to pool together the data to be analysed. This has been a major stumbling block in getting data to work for the business.”
He adds: “It makes far more sense to have a solution that can access the data wherever it sits, going to the data instead of you taking the data to the solution. By doing so, the solution can use all of the data across the enterprise, regardless of where it sits and who it belongs to.”
He’s proposing a truly technology-agnostic solution that works across platforms, vendors and applications. “It has to be fluid enough to be able to traverse the enterprise to where the data is.”
Robinson says companies need to ask themselves what relationship they want to have with their data. Do they want to make money out of it, or use it to make better business decisions or to reduce risk or even find new customers?
The world of business as we know it is changing. We have non-traditional companies offering banking services such as medical aid providers and retailers. This makes continuous evolution critical to survival, and data is the key to growing the business. Robinson continues: “Originally, big data always fitted under the IT hierarchy, but technologists didn’t know what to do with the data, that’s very much a business decision. This is why we’re seeing the evolution of the CDO within an organisation to make the technology world interface with the CEO. The CEO just wants the business to grow, and that requires creating new use cases that didn’t existed before.”
The world is definitely changing for OEM vendors, software providers and big data providers; their customers are also changing. Banks are rewarding customers based on what they buy, privacy is shifting as people allow their providers to track their activities in the interests of earning loyalty points and rewards. All of this is based on big data analytics.
The bottom line, says De Menezes, is that every single company has digital transformation as one of their top boardroom business requirements. They all want to use the data to make better business decisions, cut costs and bring better products to market. This initiative isn’t being driven by technology, it’s being driven by business demand and the necessity to stay relevant to customers and not be marginalised by new and innovative businesses. Businesses have to digitally transform simply to stay relevant and survive.
The path to digital transformation begins with automation across the enterprise, and part of that is the ability to intelligently use your data across these multiple data lakes that you've been creating for so many years. Companies need to implement automation with analytics, intelligence and self-learning that will provide an enterprise-wide solution across all of your data, regardless of where it sits, how it sits and on what it sits.
To find out more about how you can manage your data better, download this white paper:
Attend Experience Micro Focus Universe 2019 on 29 August 2019 at Inanda Club, Sandton, and find out everything you need to know about our DevOps, hybrid cloud and mainframe, security, risk and governance solutions directly from our customers. The theme for this year is “powering digital transformation”. We will support you on this journey with the critical tools you need to build, operate, secure and analyse your entire enterprise.