This is exactly what BrandsEye, a Cape Town-based sentiment-driven data provider does. According to the CEO, JP Kloppers, the company uses a proprietary mix of search algorithms, crowdsourcing and machine learning to mine online conversation for sentiment and the topics driving that sentiment. "We strongly believe what people feel today, influences what they do tomorrow, and this has really driven us to really understand how humans really feel. Our approach is that customer satisfaction and/or sentiment analysis is not a big data problem but rather a human understanding one at its core… and unless that comes first with the big data serving the objective of how we understand people. We'll always be coming up with the wrong answers. It's usually easy to understand one person, but when it comes to understanding thousands of people, understanding what's driving the human sentiments of that particular group becomes a difficult challenge to overcome. "
Speaking at the ITWeb Business Intelligence Summit 2017 yesterday at The Forum in Bryanston, Kloppers explained that an internal study comparing algorithmic results versus their human crowd verification system showed that the crowd verification system is more accurate. "Algorithm machines do not understand sarcasm, humans understand humans." One must remember that with AI, any machine that is running this system, generally requires training data to start.
"People earn money from rating data on our platform, so we have crowds all over the world (about 12/17 languages) and essentially this is a local people in the different markets that we operate, who earn money from training our algorithms and how to understand how local people feel," he explained.
Kloppers added that the main reason for the ‘lack of understanding' is the nature of conversations on social media. "Typically, social posts are full of sarcasm, slang, local vernacular and innuendo. When one adds in the difficulties around relevance when brand names can have multiple different meanings, there is a high probability that as a business you are acting on inaccurate data."
This is where a crowd-integrated approach supersedes commonly used methods of analysis, he says. "By combining the context to conversations that only people can provide, with artificial intelligence (AI), one can increase the accuracy of the data up to 97% – impossibly high when relying solely on AI."
According to Kloppers, there are three main points to consider when analysing social data. Firstly the company needs to ask whether or not this relevant to your business or research. The second, which he says is possibly the most complex, is sentiment measurement. Thirdly, he cites topic analysis: "This offers an additional layer of insight to your data and can provide granular insight into the reasons why people are expressing sentiment. This will offer a clear indication of what sub-topics constitute both the positive and negative conversation themes," he notes.
Speaking on commercial applications of accurate sentiment-driven data, Kloppers says it would enable enterprises to do accurate industry benchmarking, to have better insights when developing new products, aid with corporate reputation management as well as having accurate algorithmic trading input.
BrandsEye has been at the forefront of this technology having correctly and accurately been able to call both Brexit and Trump's victory, proving that meaningful, predictive insights can be gained from online conversation.
With the most recent Trump immigration order, Kloppers says analysis shows majority of Americans against the proposed Trump policies – such as the border wall and the so-called ‘Muslim ban' – contrary to findings of traditional polling. "Data was collected from mid-January into early February, and paints a vastly different picture for Trump as compared to the polls. Results show that currently only 9% of Americans are coming out in support of Trumps policy on the wall, and only 1% on the immigration policy. Obviously this is still an issue in progress, but what these results tell us is that there is a definite change in the fabric of the US people," he concluded.