Subscribe

Plan data warehouses. Right


Johannesburg, 01 Oct 2014

In my last article on data warehouses, Data warehouses. Right. Now, we discussed using Wherescape RED, the world's only integrated development environment for building, managing and renovating data warehouses, now I want to take a step back and understand how do we get the planning right - how long is it going to take?

You have a pretty good idea about what you want; you know where to get the data from.

But how hard is it going to be? How long is it going to take before your data warehouse development can add value? Can you actually get the data you need? Do your users really understand what you plan on delivering? What compromises will you have to make? What will your data model actually end up looking like when you populate it with real data?

WhereScape Data Driven Design (WhereScape 3D) is data warehouse planning software. It helps you explore and understand your data warehousing project at the time you need it most - the beginning.

Data warehouse planning is more than just coming up with a design and a project plan. You know the real-world issues that can impact development: source data not available at the right grain, bad data, difference in data between the proof of concept and production, socialisation of requirements ... the list is long. It is about understanding the real world constraints and issues that will affect your project - before they impact the scope, duration or quality.

WhereScape 3D helps you accurately set expectations about cost, duration and complexity of your next project. You can explore the unique characteristics of your environment, and factor them into your planning process.

Whether a team is starting with a set of source systems, a purchased industry data model, a long-in-the-tooth data warehouse or mart slated for replacement, or a complex migration to a new source system environment, WhereScape 3D starts where the team starts, providing tools and templates focused on supporting the use case at hand.

With Wherescape 3D, designers and planners can:

* Discover, profile, explore and document any potential source system for a data warehousing project, including detailed examination of source data at the table and column levels
* Implement, test and populate any purchased enterprise data model in the planned target environment
* Test planned schema, populated with real data, for functionality and coverage with end-users, before building activities commence
* Capture user stories, interviews and all other requirement artefacts as an integral part of the planning process, in a managed repository
* Generate complete project documentation, for governance, funding or internal communications
* Design, populate and test any target data warehouse schema, whether normal form, dimensional, data vault or hybrid
* Perform a complete source-to-target mapping between profiled and documented source systems, and designed target schema
* View, manipulate and associate conceptual and logical views of the proposed data warehouse or data mart
* Capture the design rationale for the new environment, while the project team is in planning

Build using a use case-driven model, implemented as templates that can be imported into, and exported from, the planning environment, WhereScape 3D allows organisations to assemble highly-specific planning tools that standardise planning activities into templates, which can be shared across planning teams, or even across organisations.

WhereScape 3D discovers and profiles a wide range of data sources, including: Oracle, SQL Server, DB2, Teradata, Sybase, Informix, Progress, postGres and mySQL.

Ultimately, WhereScape 3D's comprehensive and context-based approach to planning data warehouses and data marts allows data warehousing teams to reality-test their projects in every relevant dimension before the build process begins ? planned data warehouses right, now.

Wherescape RED consumes the design validated in 3D, it builds the semantic and view layers and is able to iterate at the speed of the business.

To understand the speed and ability of a complete Datawarehouse solution - in fact the world's only integrated development environment for building, managing and renovating data warehouses, we can have a look at the output of a Wherescape POC performed at a large retail organisation in the UK:

* Converted Logical DataModel to 3Normal Form Physical DataModel and dimensional model with type 1 and type 2 dimensions
* Loaded 3NF and dimensional model with 47 million sales transactions in seven minutes
* Demonstrated a complete life cycle loop by loading the developed model into 3D as the new master, highlighting structural differences made during the build
* Built all code from scratch - over 10 000 lines, all standardised
* Enterprize DataWarehouse load routine developed in 24 man hours
* Dimensional model built and load routine developed in 6 man hours
* Entire system fully documented
* Load routines automated on schedule
Time taken: two consultants - four days

Currently this retail organisation has retired the ETL process and is using 3D and RED to deliver in 30 day SPRINTS, replacing what traditional ETL processes had failed to deliver in 2.5 years with a Wherescape methodology to turn-around delivery in six months.

What the experts are saying:

"WhereScape 3D is VERY impressive data driven design. Obviously developed by real data modellers. 3D is a complete documentation of the entire design. So impressive...." - Claudia Imhoff, Intelligent Solutions.

"WhereScape 3D has great potential to be the default DM tool for the BI market. I get the sense it was actually developed by people that know data modelling. How refreshing."

With more than 600 customers active, WhereScape customers are able to build data warehouses with fewer people, few if, any diversions, missed deadlines or unmet user expectations, and in record time: in days or weeks, instead of months or years. Wherescape enables an agile development approach, lowered risk, pragmatic data warehousing, in-house development and a far lower cost of development.

For more information, please visit www.ixiaconsulting.co.za

Share

Editorial contacts