Johannesburg, 12 Dec 2018
Without question, data continues to grow in volume, value and velocity. It is now a critical element in gauging the worth of an organisation, but also key to helping your company thrive in the new digital economy. Organisations today are also being compelled to adopt modern applications, with new functionality driven by both business and user demands.
Although data is perhaps one of an organisation's most valuable assets, it can sometimes be forgotten amid the excitement of exploring innovative functionality in a new system. Yet, the migration of data is often considered last. IT analyst Gartner states: "Through to 2019, more than 50% of data migration projects will exceed budget and/or result in some form of business disruption due to flawed execution."
What Gartner says may seem like a staggering figure, but maybe not so surprising when you consider that many IT departments carry out very few data migration projects and so may not have the skills required. At the same time, the complexity of a data migration project, particularly when migrating from multiple systems, is so often overlooked.
Barry MacDougall, Service Delivery Director and Data Migration Expert at JMR Software, suggests: "There are many things that can contribute to data migration project failure, including late planning, but all too often in projects, the 'happy path' is plotted. This generally involves perfection in every element, including the use of small amounts of high quality and complete data; yet taking this approach creates many problems later on and is why comprehensive testing and reconciliation of data is required."
We all know that any IT project has an element of risk, but none more so than data migration. A project that fails to deliver can lead to reputational damage, both personally and corporately, as well as dissatisfied employees and customers and a new system that struggles with user acceptance because it is not delivering what is required.
So, how can you mitigate data migration risk?
Risk is often a result of unknowns. Yet, these are often only identified when the project is mid-flow or even as late as user acceptance testing (UAT). Data migration can be complex, particularly if you have multiple data sources of varying quality, and with data sitting on different IT environments. Making sure you have the right skills is critical to a project's success. So, too, is carrying out a detailed scoping and mapping exercise to properly understand the extent of the data you have, but also working out how much needs to be migrated.
Gartner analyst Ted Friedman has said previously: "There are many things that need to be considered to reduce the risk in data migration; you could limit the number of data sources, but you also need to leverage from existing knowledge. A phased migration should be considered and only migrate the data that is needed. Partitioning data in phased migrations reduces complexity, and finally, focus early and heavily on data quality."
The more you know about the data the better, and involving business from the start will help considerably when it comes to considering the best strategy for the data migration: should it be phased or big bang? Again, check whether you have the best resources, skills and technology to do this in-house. You may come to the conclusion that your resources are best deployed in new product configuration or other development work, and it is better to outsource your data migration.
Engage business early to gather requirements
Moving forward with a data migration, it is essential to work with the business users to fully understand the data, its location, its state and establishing what needs to move. The tendency for business is to move all of the data. However, this isn't always required or in the best interest of the overall project; for example, if you only have a short window of time to load the data into the production environment, too much data could make it impossible to cut over in time.
Getting business involved early is really important, says MacDougall: "Understanding the scope of data that needs to be migrated very early on is really important. Decisions need to be made and the business has to be involved so that you can agree exactly what needs to be moved, but also if any work is required to improve the quality of data."
So, moving data can also provide business with the opportunity to tidy up poor quality data, restructuring, transforming and enriching it. Regardless of which approach you take, communication with business needs to be strong.
Reconciliation is critical for data migration success
Testing the data for quality and completeness, in other words, reconciling what is being migrated from one system to the other, is a critical step in the overall project. You may hear of projects where all of the data has been migrated into the new system, but the data quality is poor or simply can't be used in the new system.
Reconciliation and testing the data throughout the data migration process is crucial to ensure the completeness and quality of the migrated data. Typically, this may involve spot-checking data that is in the process of moving, but can end up being the 'happy data' and it is only at the UAT stage you find out the extent of any gaps or quality issues with your data.
There are extract, transform and load (ETL) tools that will compare data counts from the old system to the new, but this may not catch all of the issues. Two levels of reconciliation are advised: process and business.
He continues: "The process of reconciliation ensures that the data migration process is performing correctly. It simply counts the number of data items extracted, transformed and then loaded. For example, the number of clients migrated.
"Business reconciliation, however, confirms that the data is migrated correctly in terms of the business requirements that have been captured early on at the mapping stage. This may involve a count of the number of data items, but can also be a sum of the value of the monetary amount, eg, the sum of the total pension contributions per year. This is why business needs to be involved early on, so that we can capture the business requirements for reconciliation at the start of the project."
Quite frequently, there can be delays in data migration that impact the implementation of new systems because data issues have to be fixed manually as data is loaded. Poor quality data will seriously deteriorate user adoption of any new system and it can take a lot of time and effort to grow user confidence.
If you also see data as one of your most precious assets, then, when you need to move on to new technology, it's worth remembering the 3Rs of data migration. Understand the risks early on, get business involved from the off to button down requirements. Finally, be thorough in reconciling and testing data before it is loaded into a production environment, and that way, when the time comes to migrate, it is literally a push of a button.
Data migration is frequently viewed as an IT project that doesn't always get the time and attention it requires from the business. Yet, for data migration projects to be successful, it is clear that business needs to be engaged early and frequently throughout the data migration process to ensure this valuable asset is migrated successfully from old to new.
Share