Facing the integration challenge

A large corporation launches a new product/concept. The marketing folk anticipate Lourie awards, the sales folk plan the next incentive trip, the techies are delirious with the power and brilliance of their new toys. And the letters go out to the customers listed on one or other of the databases within the organisation.

Integration is as much a business challenge as a technical challenge.

Peter Hall, consultant, application integration specialist

The letters read something like this - "did you know that if you already have our product A, B, C or D you qualify for our amazing new concept...? If you have any of these products please contact us and we will..."

When these letters go out, are the customer databases driving the mailing list complete and up to date? Can the same customer be matched across different divisional databases with a measure of confidence? This may seem a facile example, but it can happen when the divisions that service products A, B, C and D have built their systems in relative isolation over the years. The corporation might ask "do we know our customer?" The customer might ask the far more damaging question "does the corporation know me?"

Years back, when Assembler programmers were being dragged kicking and screaming into the brave new world of Cobol, systems were built on a single homogeneous mainframe. In that environment the problems of integration were less tedious or visible. It was relatively simple to create, extract and merge programs, and ergo the data moved painlessly from system to system. We generally had a single database system and a single programming language and this of itself made cooperation easier.

Then open systems arrived, Unix, Windows, object orientation, component-based development, mergers and acquisitions complicated the process, and more and more critical computing was migrated to the desktop. Vital enterprise data became increasingly dispersed and dislocated. Architectural decisions, such as they were, devolved away from traditional IT executives and the business started to adopt technologies to satisfy specific divisional needs.

Of course it is true that many of these specific solutions met their narrow divisional objectives quite well and relatively quickly, but at what cost to the broader objectives of the corporation? How do we convert the huge amount of legacy data we have been accumulating over the years into the valuable information that helps us put a face to our customer?

When a customer initiates a transaction the enterprise needs to know about it, not just the specific division that the customer interacts with directly. If I arrange vehicle finance with a bank, the credit card division needs to know, the retail banking division needs to know and the home loans division needs to know if I`m a 'new` customer, or a repeat business customer. If I am an existing customer, then what are my transaction patterns and my track record? What financial exposure does the bank have? Am I good for all the deals that I have done in relative isolation? Which of the half dozen or so assets and liabilities statements is up to date? The process of determination should be relatively automatic to measure the risk.

The need for integration has been with us for many years and the solutions have generally involved a proliferation of home-grown programs to try to create an illusion of integration.

Then along came enterprise application integration (EAI), the pot of gold at the end of the rainbow. Suddenly manufacturers of EAI products sprung up like mushrooms, each with a slightly different twist, each insisting that they had the solution to the woes of dispersed and dislocated data. Analysts and consultants drew layered architecture schematics, each trying to outdo his chums with more layers and more complexity. IT executives found themselves drowning in a sea of new genre buzz words, buzz phrases and techno babble.

Integration of dispersed systems requires that complex decisions be made. There is no quick fix, no easy way and there is certainly no cheap way. Every corporation has different needs and there is no 'one sock fits all` solution. Integration is as much a business challenge as a technical challenge.

The corporation faces major questions about where to start, what the real drivers are, how to quantify the value of existing systems and how to determine the return on investment (ROI) of integration. These are very difficult questions, made no easier by the IT industry`s ability to adopt new fads at an alarming rate.

The responsibility of drawing the integration road map to the future and of defining the role and scope of EAI in the corporation devolves to the enterprise architect. Their role has grown out of the haphazard divisional systems development and has become pivotal. It is their task to analyse the scale, scope and style of EAI best suited to their particular corporation. They have difficult and sometimes unpopular decisions to make. It is their role to define the standards for the enterprise. While computing might not be centralised, the decisions should be.

SA may be blessed with many competent IT people, but significant numbers are leaving our shores for what they perceive to be a safer and better life. Generally these people are late-20s to mid-40s, many with young families, most of them are solid, reliable resources, and all of them have broad and marketable technical skills. The keyword is experienced. The dwindling pool of experienced talent is putting pressure on IT executives to build the power teams necessary to take up the challenges of integration.

Meanwhile, corporations continue to face the dilemma of where to put limited financial resources. Is it behind brand new attention-grabbing products or is it behind servicing existing and new customers with improved skill and knowledge? Obviously it is both.

Consider the cost of retaining a customer with better service against the cost of gaining a new customer. Unless you operate as a monopoly, you know the cost of building and retaining market share and the pain of losing it. You`ve got the data, but have you got the information? The challenge is to leverage the customer data, to create useable, valuable and verifiable customer information. This is surely the role of application integration across the enterprise.