The data warehouse environment has changed significantly in recent years, but even bigger shake-ups are to come, says Barry Devlin, founder and principal of 9Sight Consulting.
Devlin, described as a founder of the data warehousing industry and among the foremost authorities worldwide on business intelligence (BI), will be a speaker at the ITWeb Data Warehousing conference in Midrand next month.
He says the data warehousing environment has seen a lot of change in recent years. “There are many differences from five years ago. However, probably the most obvious and important is the significant move towards ever more timely data.
“Today, many warehouses are being asked to support substantial volumes of near real-time data. These needs drive very different technical solutions that those that were common five years ago.”
However, the most significant changes lie just ahead. Devlin says: “The most significant change I see coming is in impact of commodity memory and processor technology on database design and use.
“We've already seen some of the results of this in analytic appliances, but the widespread use of in-memory databases and parallel processing in multi-core CPUs will have enormous impacts on how we architect data warehouse environments.”
He notes that most big companies are still grappling with the legacy of multiple warehouses and marts they have built over the past 10 to 20 years. “These systems often contain valuable but overlapping information and have committed users. However, they are often based on out-dated platforms and are expensive to maintain.
“New requirements are difficult to satisfy due to conflicting data sources, complex analyses and costly solutions.”
Devlin says the first step in overcoming this is to revisit the company's BI strategy; asking why multiple copies of the same information exist, and how the number of existing copies can be reduced. “A new strategy that severely limits the number of additional copies is then needed.”
Don't miss Devlin's keynote address: 'Is there life after data warehousing?' at the ITWeb Data Warehousing conference at Gallagher Estate on 30 November. Click here for more information.
Share