New data architectures needed

BI and data warehousing are continuously developing; it's up to businesses to become a part of this evolution or run the risk of falling behind, says Rick van der Lans.
BI and data warehousing are continuously developing; it's up to businesses to become a part of this evolution or run the risk of falling behind, says Rick van der Lans.

The architecture of data warehouses needs to change in order for businesses to effectively move towards more agile business intelligence (BI).

These were the sentiments of BI expert Rick van der Lans, who presented a two-day seminar in Bryanston this week. According to Van der Lans, the world of BI and data warehousing are continuously developing and it is up to businesses to become a part of this evolution or run the risk of falling behind.

In the past, BI users would accept that data was a little outdated, but current BI users want live data and to see what is happening in real time, said Van der Lans, a trend termed Operational BI. To demonstrate this kind of BI, Van der Lans used the running of the Rotterdam Harbour, in the Netherlands, as an example. The largest port in Europe, the harbour has very narrow entry and exit points, making the scheduling of when boats move in and out of the harbour vital. These schedules are run by a single company that has to control 100 000 records per second in order to prevent bottlenecks or collisions. "This is an example of BI working effectively today," said Van der Lans.

According to Van der Lans, agile BI is key, as statistics show that businesses have less time to make decisions, but that it is taking more time for reports based on new data to be produced. The result is self-service BI, with business users more commonly bypassing the IT department and simply doing the tasks themselves.

"Vocal, demanding and influential users are increasingly driving BI purchasing decisions. They're choosing easier-to-use data discovery tools over traditional BI platforms," according to Gartner.

Gartner predicts that business units will soon control at least 40% of the BI budget, taking the control away from the IT department. "This is happening because the data warehouse architecture we built was fine in the past, but it's just too inflexible. Businesses must look to new technology," said Van der Lans.

Data warehousing

The beauty of data warehousing, according to Van der Lans, is that the business has one big database where information can be cleaned up and data can be coded correctly, making for more efficient decision-making. The problem, however, is that these data warehouses must be upgraded regularly to match the changing needs of the enterprise.

At present, most data warehouses are made up of a chain of databases. "This architecture has served us well for the last 20 years, but is it agile enough today?" Van der Lans asked. "Should the user need to change something, this chain architecture is not designed for flexibility and speed, which are necessary in today's marketplace." For Van der Lans, the fact that things need to be cheaper, faster, more flexible and capable of handling more data is one of the key challenges facing the database world.

What does this mean in SA?

"In countries such as South Africa, it feels as if we are getting close to the turning point in terms of data and BI. We're starting to understand that we have to react. It looks as if we are slowly getting there, and I think by 2013/14 things will be different in this space," said Van der Lans, referring to when he thinks South African businesses will fully embrace BI and data warehousing. He added that more and more companies are realising that the tools they have been using for the past 20 years need to be updated as technology changes.

Quoting a Dutch saying, "What the farmer doesn't know, is what the farmer doesn't eat", Van der Lans believes that the issue for many businesses when it comes to BI and data is that they have heard about all the technologies, but they have never properly experienced them and thus are unable to conceive their true benefits.

He advises that businesses do a proof-of-concept to better understand what these tools do and how they can improve efficiency.

See also