In application development, many people use so-called agile methodologies for delivering short four- to six-week 'bursts' of work. These cycles, referred to as 'sprints', result in some deliverable work which can be put into production and integrated into the business at the end of the 'sprint'.
However, in the business intelligence (BI) world, Ralph Kimball and associates have long been advocating that a data warehouse solution comprises a number of business-process-related cycles.
The landscape of this implementation is the Bus Matrix, which maps out the business processes and their related conformed dimensions. The data warehouse can then be created holistically by delivering each one of these business process's data.
In this way, one can build the collective data warehouse in smaller iterations, as long as the conformed dimensions that make up the information 'bus' are properly maintained.
Kimball and associates maintain that such a building block, if sourced from a single source, should not take more than three to six months to complete. This means that after six months, the first deliverable should be available, in production, and actively being used by end-users. This concept is counter to the popular belief that such a project will only deliver an entire data warehouse, and that such an effort will take years to complete.
In contrast, this method of breaking the 'whole' up into tangible and implementable pieces and then implementing these pieces in relatively short time-frames, lends itself well to the philosophies of the agile methodologies. One can start isolating the business process implementations from each other and work on implementing them in three- to six-month 'sprints'. This will get the bigger data warehouse on the road quicker, with shorter time to wait between requirements and delivery.
The toolkit
This concept is counter to the popular belief that such a project will only deliver an entire data warehouse, and that such an effort will take years to complete.
Cor Winckler is technical director at PBT.
Typical tools that are used in such agile implementations are also relevant. Using shared online task tracking, for example, will allow teams to potentially work in a distributed organisation, but still deliver a central implementation. Sharing knowledge in project wikis and making use of centrally maintained (and versioned) source control are also key to the agile approach and again translates quite well into the BI paradigm.
Delivering in short bursts fits the Kimball data warehouse lifecycle (nowadays called the Kimball Lifecycle) very well. It forces the entire cycle to come full circle, and revisit the bigger picture every time a new business process is tackled.
It allows for simultaneous parallel development of multiple business processes, as long as the teams are communicating about the integration points, which are typically the conformed dimensions of the bus matrix. This kind of communication is embraced by the agile methodologies and becomes evident in the use of online share issue/task tracking and knowledge sharing through project wikis.
The repetitive nature of the process also makes it possible to progressively improve the end project with very little, or sometimes even no impact, on the existing end-user base.
The end result is an ever-improving, ever-evolving data warehouse which the users are exposed to from a very early stage, making user adoption and proof of value much easier than it would be in multi-year projects. Furthermore, it is also much easier justifying a short six months of initial activity before a usable deliverable is put down, than to secure budget up-front for a three-year project.
While in essence it is the strong business drive and vision that should justify the BI investment, it is important that the organisation as a whole acknowledges the solution as an information enabler, able to add value to any area of the business if used correctly.
The hard, measurable benefits highlighted by such 'sprints' are certainly important to justify the ongoing BI project. Such cycle development is characterised by an emphasis on people, communication and responding to change - one that is able to drive the vision of the organisation, add value to the departments that need it and ultimately achieve desired ROI overall. It's time to get on the BI-cycle and get the wheels rolling!
* Cor Winckler is technical director at PBT.
Share