Exploding data volumes and increasingly complex data analytics requirements are driving a need to simplify data management.
So says international analytics and data warehousing consultant Michael Spanoudes, who consults to analytic data solutions firm Teradata. Spanoudes says new analytics requirements are characterised by complexity and speed, with an increasing need for access to all data across the entire enterprise.
"The complexity of analytics continues to grow as we add new dimensions - including temporal and geospatial data - to analytics. There's greater adoption of analytics across the enterprise to better understand individual customer behaviour, characteristics and trends," he says.
"Data volumes continue to grow at an increasing rate as businesses extend their customer interactions to mobile devices and leverage social media tools, and we continue to add new and diverse data such as unstructured data from the Web, social media, GPS co-ordinates and machine-generated RFID, which drives the need for new analytic techniques."
This rapid change, he says, is catalysing a need to review data warehouse architecture.
"The next-generation analytic environment must be able to support a wide range of analytic data and techniques, or else you end up with analytic silos," he says.
The next-generation model needs to deduplicate data and processes to allow for better data mining, to answer more complex, cross-functional business questions faster, he says.
Spanoudes adds that many enterprises face a situation where their data resides in hundreds of data marts, impacting not just the platform and disk space, but also data movement.
"This becomes unmanageable," he says. Collocation of data marts into a single server may solve part of the problem, but it still does not eliminate the duplication and data-movement issue to refresh these proliferated data marts, let alone the lack of ready access to cross-functional data because of the siloed and summarised nature of the data in the data marts, he says.
"When data is spread everywhere, it is difficult to find answers to questions that are critical to understanding business performance and making decisions that affect future enterprise competitiveness," he says. In some cases, businesses may not even be aware of the complexity of the questions they could answer, once they have immediate access to all the available data residing across the enterprise.
With logical integrated data models, in a detailed, historical and normalised structure, all data is loaded once and centrally stored, and is used many times across the whole enterprise to answer complex questions. This enables the enterprise to maximise the business value of the data.
Spanoudes cites Kevin Strange, VP, Gartner Group, as stating that "enterprises initiating data mart consolidation efforts in 2002 that use quality methodology and application-neutral data warehouse implementations would experience a decline in spending of at least 50% and an increase in business value of at least 500% by 2004 (0.7 probability)."
Rearchitecting the data model does not necessarily have to take a big-bang approach, Spanoudes says. Enterprises can focus on areas or departments with the most business needs, and address those first. "They can build the correct infrastructure for key departments, then retire the corresponding data mart.
"They can then move to truly integrate more and more of the data in specific data marts until they are in a position to retire those servers. Over time, they end up with the next-generation integrated enterprise-modelled data, and no 'questions from hell' if they have the right performance from the database engine to support concurrent complex queries against the normalised data-model structure," he says.
Spanoudes will elaborate on analytics and data warehousing models at the upcoming ITWeb BI Summit. For more information about this event, click here.
Share