The volume, velocity and variety of big data means that if a business can't filter and analyse it effectively to ask the right questions in the first place, it may be swamped with information.
This is one of the challenges of managing big data effectively, says Craig Stephens, principal solution manager at SAS.
"Big data is not just about huge data sets," he says. "The velocity and variety of it is what challenges enterprises."
Stephens says the main challenges are threefold:
* The acquisition of the data - sophisticated systems are needed to acquire the data, and much of it resides externally.
* The data must be stored in a cost-effective manner - commodity hardware and storage can be leveraged to address those costs.
* Then, exploiting that data - there is no point only acquiring and storing data; businesses need to derive value from it. Insights derived from the data need to be implemented into a business process.
The value of effective big data management and analysis is potentially bigger than the volume of the data itself. Stephens points out that companies now have the ability not just to understand customers and their environments, but also to predict future outcomes.
"There is a huge range of applications for big data," he says, "from retail, to healthcare, utilities and mining."
For example, he says a data scientist working for a mobile operator may want to predict prepaid customer churn based on the frequency of their airtime purchases, phone usage patterns and even what service providers their friends prefer. With advanced analytics in place, it is possible to create the required customer segmentation model.
"The challenge is that he may want to access 1 000 different attributes to build the model, when perhaps only 200 attributes would be required for that model. This is where analytics comes into play too - the data scientist can use advanced analytics tools to access the 1 000 attributes, determine what he needs, and then filter out only what is relevant to create his model.
"Traditionally, organisations would bring in all the data and use all data sources. Analysts may spend 80% of their effort getting access to data and building their models this way, which is expensive and counter-productive, because they are doing operational-type work instead of delivering insights with real business benefits.
"But it isn't necessary to break down an elephant to get a small component. Our approach is to use analytic techniques and profiling to determine the relevance of the data, putting analytics into the management of big data.
"Analytics benefits business throughout most of the big data management phases - in acquisition, management and exploiting the data. It can be used to build rules to run an analytical model to determine the correctness of the data, for example."
Stephens expects big data to play a critical role in future business success, if it is managed effectively. Even people who aren't analysts or who don't understand analytical techniques should be empowered to ask questions of big data, based on their own expertise, and quickly and easily find anomalies and get answers to questions they haven't even thought of yet.
"Businesses need to start utilising big data in their everyday world, instead of throwing it to IT to 'look into'. They need to start moving forward, effectively manage it, use predictive analytics and operationalise it for real business benefit," he says.
Mark Torr, director of SAS Technology Practice, will address the ITWeb BI Summit next week on key developments in the analytics space. For more information about this event, click here.
Share