Big data has been a hot topic for some time now, and unfortunately, many big data projects still fail to deliver on the hype. Recent global studies are pointing out that it's time for enterprises to move from big data implementations and spend, to actually acting on the insights gleaned from big data analytics.
However, turning big data analytics into bottom-line benefits requires a number of things, including market maturity, the necessary skills and processes geared to auctioning insights. In SA, very few companies have these factors in place to allow them to benefit from significant big data projects. Despite the hype about the potential value derived from big data, in truth, value derivation is still in its infancy.
Locally, early adopters have been major enterprises like banks, where big data tools are necessary for sifting through massive volumes of structured and unstructured data to uncover trends and run affinity analysis and sentiment analysis. But, while they have the necessary advanced big data tools, I often find these new technologies are delivering little more than a sense of confirmation, rather than the surprise findings and bottom-line benefits these enterprises have hoped for.
Exercise in futility
This may be due to processes that result in slow application of new insights, as well as to a dire shortage of the new data science skills that marry technical, analytics and strategic business know-how. Currently, the process of big data management is often disjointed from start to finish: companies may be asking the right questions and gaining insights, but unless these insights are delivered rapidly and companies actually use the insights effectively, the whole process is rendered ineffective. There is little point in having a multimillion-rand big data infrastructure if the resulting insights aren't applied at the right time in the right places.
The process of big data management is often disjointed from start to finish.
The challenge now is around the positioning, management and resourcing of big data as a discipline. Companies with large big data implementations must also face the challenges of integration, security and governance at scale. And there are many misconceptions about big data, what it is, and how it should be managed. There is an element of fear about tackling the 'brave new world' of technology, when in reality, big data might be seen as the evolution of BI.
Most commonly, companies are feeling pressured to adopt big data tools and strategies when they aren't ready, and are not positioned to benefit. As with many technologies, hype and 'hard sell' may convince companies to spend on big data projects when they simply are not equipped to use them.
In SA, only the major enterprises, research organisations and perhaps players in highly competitive markets stand to benefit from big data investments. For most of the mid-market, there is little to be gained from being a big data early adopter. Already, cheaper cloud-based big data solutions are coming to market, and - as with any new technology - more of these can be expected to emerge in future. Within a year or two, big data solutions will become more competitively priced, simpler, require fewer skilled resources to manage, and may then become more viable for small to mid-market companies.
Until then, many may find more effective use of their existing BI tools, and even simple online searches, meet their current needs for market insights and information.
Unless there is a compelling reason to embark on a major big data project now, the big data laggers stand to benefit in the long run. This is particularly true for those small and mid-size companies currently facing IT budget constraints. These companies should be rationalising, reducing duplication and waste, and looking to the technologies that support their business strategies, instead of constantly investing in new technology simply because it is the latest trend.
Share