Data warehousing is a $100 billion industry and is growing, as information has developed from being a scarce resource into a super-abundant one.
So said Bill Hoggarth, head of business intelligence (BI) at CQS Technology Holdings, giving an overview of how information is being used in business today, during yesterday's ITWeb Data Warehousing 2010 conference at Gallagher Estate in Midrand.
Hoggarth said future data warehouses will need to evolve to manage the explosion of information, as technology becomes more advanced. “The Hadron Collider generates 4TB of information every single second.
“The electronic footprint is continuously getting bigger and business has become incredibly complex because there is no certainty anymore. Consumers have so much choice today that there is no such thing as captive market anymore.”
Data explosion
According to analysts, unstructured data or soft information such as videos, images and documents are increasing in volume by more than 60% year-on-year. However, data warehousing project budgets are failing to grow at the same rate.
This means, according to Hoggarth, there is more data in the world than organisations can store and analyse to get an insightful and accurate view of their business.
The structured data world is 5% of the total pool of information, according to Hoggarth. “And as data architects, one of the things we are not comfortable dealing with is the unpredictable.
“The debate means nothing to business users who just want information; they don't care where it comes from. Organisations need to start thinking about what information is required to support a decision.”
Information delivers insight
Hoggarth explained that data should be considered as a primary asset next to capital and people, adding that information will emerge as an executive-led initiative and the role of information will fundamentally be managed from top-down.
“We have a crisis of identity in the BI world. We need to think hard about the purpose of the data warehouse we are putting in today. If it's about enterprise performance management, operational reporting, governance and compliance; they all have an impact on how data professionals manage their information assets,” noted Hoggarth.
Hoggarth said more information from data warehouses is being used to build business models. He pointed to Amazon, which has built its entire business based on customers' buying
Share