BI`s next frontier: social computing and the cloud

As almost everyone in IT knows, business intelligence (BI) is all about delivering the right information to the right people. But these days, this information includes data both from unstructured and semi-structured sources.

Some unstructured data, including e-mails, SMS, blogs, corporate documents, and even voice, could include vital information relevant to the decision-making process and, as such, to the BI environment. This makes an already challenging task all the more complicated, according to Charl du Toit, BI Solutions Manager, EOH Oracle Services. BI systems have traditionally not focused on unstructured data, leaving this data to knowledge management (KM) systems.

"As a result, these two areas are starting to merge; but, as most BI practitioners know, unstructured data access brings unique problems. Scanning technologies help to ensure that most information, even if it is paper based, is now available electronically, but technologies to then effectively parse this data and extract meaningful information from are still maturing," he adds.

Manual processes are often used to tag documents with metadata, but automating this process effectively would really bring unstructured KM into the BI realm. Such a process would parse documents and deliver an output of some sort (text file, XML, DB table) with tags and data elements extracted from the unstructured data, which can then easily be integrated in the normal BI environment. Services-oriented architecture (SOA) can also assist in this case where information is tagged at time of capture and fed to the BI environment.

Of course, another approach would be to just deliver the unstructured content as is during the BI final reporting stage, but then there is no analysis or integration with other information - just the unstructured content as is. "Some processing to deliver the data within the context of other structured data would be better than nothing and would still add value," maintains Du Toit.

For many a corporate, social computing is perhaps the ultimate in unstructured data. While the use of social computing in corporate intranets and in other medium- and large-scale business environments is still in its infancy, it certainly could have benefits for BI, provided that relevant information can be distilled from these sources. It also depends on a company`s adoption of social computing as an integral part of doing business.

"Social computing in some form or other is already being integrated into enterprise resource planning [ERP] systems, among others. It could add value by leading to better communication, effectively mimicking the informal social networks of employees. If a social network gets extended to customers, suppliers, and shareholders then this network could begin to house vital information. The challenge, yet again though, would be to tag and extract relevant and useful information from this network, explains Du Toit.

Perhaps vendors of social computing systems could adopt a services-oriented approach, which might make it easier to identify users, topics, and relevant elements, and extract these into the more formal BI environment, he suggests.

Speaking of software as a service (Saas), there is no reason why most - if not all - BI functions can`t be done `in the cloud`, says Du Toit.

Certain BI applications are well suited to the cloud. Take for example applications for small companies that do not have a large amount of data, or those applications for which the source data already exists in the cloud (think Salesforce.com), he says.

There is also the obvious benefit of moving BI to the cloud in that there is no need to invest in expensive hardware architectures, as well as possibly much quicker deployment times by utilising predefined application logic and infrastructure without having to define and implement it yourself.

However, there are some constraints to the cloud, which could make BI functions near impossible. For example, moving large volumes of data into the cloud might be problematic due to latency and data storage fees.

"Data security issues might also prevent certain companies from moving BI to the cloud. However, as speed becomes less of an issue, storage capacity becomes almost infinite, and cloud providers become more mature regarding security, we might see more and more BI solutions being made available in the cloud, especially for the SME market," surmises Du Toit.