Subscribe

Web services meet data integration

The one constant in the Web services market at the moment is that there is no clear definition of what exactly a Web service is, or what a Web service should consist of.
Julian Field
By Julian Field, MD of CenterField Software
Johannesburg, 28 Mar 2003

Defining exactly what a Web service is can be complex. Broad definitions exist, but no standards body has laid down the law, although many people are trying.

A broad definition will include business-to-business application integration, data integration, system integration and Web applications combining to form a community of systems that pass data between each other.

Web services are a type of federated trust model in which disparate applications and data have to interact and talk to each other.

Julian Field is MD of CenterField Software, the local distributor of Ascential Software`s solutions in SA.

There are two enablers to Web services: Java or .Net, another religious battle on the rise; and security, a vital aspect of the Web services environment. Web services are a type of federated trust model in which disparate applications and data have to interact and talk to each other. They are going to play a vital role in the data integration market sector: in real applications delivering real user value.

Routing example

A simple example of an application making use of Web services integrated with data integration can be found in a routing application. This application has the seemingly simple task of directing the user from one location to another in the shortest possible time. All the application needs to do is locate the user`s position, obtain the required destination, work out how many alternate routes there are, compare the estimated time to travel each route and then provide the user with the fastest route.

It`s not as easy as it looks, and this is where Web services standards can ease the process. First, the current location needs to be determined. Either the user enters it or the application obtains data via a global positioning system, with Web services providing the mechanism to obtain the data without any middleware or additional development costs (assuming the third-party data is Web services-enabled).

Next, data needs to be downloaded from a mapping company to determine the two locations and the various routes between them - again Web services simplifies the process of getting and understanding the data. Then data from various other systems, such as weather or traffic reports, needs to be gathered. All this information is then collated to warn the user of impending problems and to advise on alternate routes.

The traditional method of development would see applets written to communicate with different applications and convert different data formats into something understandable to the routing application. This would need to be done for each unique application and data set. With Web services technology as the communications standard, all the work is already done apart from a request to the various systems for data.

Web services also impact the output of this application. The data can be sent to multiple devices, from cellphones to PC monitors, and the software on the device`s browser interprets the data to the appropriate format - even voice in the near future - all due to Web services standards.

Data quality and cleansing

Another scenario, assuming that all security and trust issues are resolved, involves data integration on a broader scale. Suppose Joe Bloggs wants to open a new bank account and does not mention any existing accounts or overdrafts. The bank wants to check up on his previous accounts before allowing him any leeway and therefore launches an online Web check - using Web services to communicate and gain access to the relative data from other banks.

The problem here is that the answers to the query could include Joe Bloggs, J Bloggs, JD Bloggs and Joe Blogs. This is where the more traditional role of data quality and cleansing enters the Internet age. A data integration company - for lack of a better term - will pull the data down to its own servers, and sift and clean it to ensure only the right Bloggs data is returned to the bank.

This idea can be extended to business intelligence environments as well. If an organisation wants to populate a data warehouse, for example, it does not need to employ the skills and buy the infrastructure necessary for the extraction, transformation and loading (ETL) process. It can hire a company online to upload the data, perform the ETL and then return the clean data.

These ideas are not dot-com smoke and mirrors, but the technology already exists to make it a reality. Web services technology allows a third-party to access and confirm a corporation`s data integrity and quality online as one of the first steps before any analysis or reporting is done.

The results of combining the business value of data integration and the technology of Web services should be less human intervention from data processing tasks and reduction of associated errors. The promise of the Internet and Web services will also lower the barriers of entry data to integration services and permit smaller concerns to have the same access to data manipulation tools as their enterprise cousins.

Share