The importance of open data

Data can be an organisation's greatest asset. It is the key to understanding and managing performance. Used effectively, it tells you what you must do to succeed, how you are doing relative to plan, and why.

It enables better business decisions that drive performance, and builds the competitive advantage that ensures success. But reaping the benefits of data is not easy.

Data is typically fragmented. It is often incomplete. And it is not readily available in a form that can be used effectively by the people who need it. To realise its benefits, data from a variety of sources must be turned into information that can be used consistently across departments and divisions. Only then can business questions be answered, business issues solved and better business decisions made.

The key to realising the potential of data lies in implementing an open data strategy. An open data strategy for business intelligence (BI) lets organisations consolidate data from any source, with any latency (the time required to make needed data available), to create a single business view of the enterprise that can be used to drive better performance.

To be effective, information must not only be timely and accurate, it must present a single, consistent, enterprise-wide view of reality across departments, divisions and corporate functions. However, to provide this information, organisations must convert vast amounts of data from day-to-day operations. Faced with an increasing number of data sources, exponential growth in data volume, data integration issues and the need to combine historical and real-time data for operational intelligence, the task of converting data to usable information can be daunting.

Typically, the sources of data span multiple systems, platforms and underlying technologies. Many of these have evolved over time, often without rigorous central management. Business constraints have made it difficult or impossible to rationalise these into a single, cohesive data source. In addition, the time and money invested in these systems means it is seldom practical or cost-effective to migrate the accumulated data into a single environment. An enterprise strategy for accessing data must thus be put in place to allow users to access data no matter what its source.

Perhaps the most immediate impact of data proliferation in the enterprise is the proliferation of the tools and systems needed to access different data sources. This proliferation of tools - and the associated costs of training, maintenance and integration work - is one of the key drivers of business intelligence standardisation.

To overcome issues concerning the integration of data for use with disjointed tools, it's often required to stage data in multiple places to meet all the needs of all users. This results in time lags, as more and more data must be migrated, duplicated and kept synchronised.

The solution to these problems lies with common metadata - a single business view into all data required to manage the business, regardless of where it lies within the organisational infrastructure. To achieve this, BI standardisation built on an open data strategy is essential.

Standardising information delivery on a single BI platform means less work, lower costs and faster implementations for IT departments. It means less training time and an easier, broader adoption of the fewer tools associated with a standardised BI platform. It generates greater business agility, as the integrated system allows access to all relevant data in ways not possible with fragmented BI implementations. The result is more and better business insights that drive improved performance.

The foundation for BI standardisation is data integration. If a BI system cannot bring together all of the data assets that are available, and make the data readily accessible in whatever form is required, then achieving standardisation is next to impossible. While the drivers and sources of data have changed rapidly, one thing remains constant: getting the data right, and making sure the right data assets are in place, is key to delivering information.

The costs associated with getting the right data in place are typically high - up to 80% of the total cost of a BI project. And these costs are not discretionary: laying a solid data foundation for BI trumps all other standardisation activities. If an enterprise BI application is built on the wrong data, or on out-of-date or incomplete data, the value of the system is compromised long before data reaches the business user.

Access to data is more than the simple ability to connect a reporting or analysis tool to a given data source, however. The magic lies in delivering access to data in the context of value, both from an information and business intelligence point of view, and from the perspective of cost of duplication or data migration.

An open data strategy and a fully integrated set of BI capabilities will allow organisations to reap the benefits of their data. An open data strategy will enable users to leverage existing and new investments in capturing data assets and in adopting data integration strategies, while direct data access, extract, transform and load (ETL), and enterprise information integration (EII) capabilities ensure the basics are taken care of.

A consistent user interface will promote rapid adoption of BI by users across the enterprise, not just within IT. The ultimate goal of a business intelligence initiative is, however, linked with the issue of data latency - the time required to make needed data available. Organisations have data that spans all of these latencies; therefore, the BI solution should cover them all. EII will let organisations leverage in-place data assets, even in complex data environments. The result is a federated, 360-degree view of data that includes historic data from warehouses and real-time data from relational databases. This ensures fast performance, sophisticated session caching, and a single view of all data sources.

Perhaps the most important aspect of any information delivery project is having a sound understanding of exactly what information it is you are delivering. The mechanism for delivering a common business view is enterprise metadata. Highly tuned, comprehensive metadata models turn the underlying complexities of disparate business systems into easy-to-understand business models, providing business-driven views into heterogeneous data sources, with the ability to build in common business rules, calculations and filters at the metadata level. Once defined, metadata is a valuable asset that should not be restricted to any individual application.

The final piece of the open data strategy lies in the fact that typical organisations with large-scale BI requirements have an incredible range of user requirements to satisfy. A truly open BI solution allows any user, with any information requirement, to access any data using the BI capability that makes the most sense. The ability to deliver on the widest range of requirements, independent of the underlying data sources, but based on a common business view is paramount.

For businesses implementing enterprise-scale BI applications, an open data strategy is thus not just a nice-to-have. A truly open data strategy will result in a platform for managing performance and driving corporate success by allowing organisations to access any data source or combination of data sources, develop common metadata across them for a common business view, and then leverage that common business view to securely deliver any business intelligence capability to any user.