MQSeries.net :: View topic

JohnMetcalfe · Posted: Mon Sep 05, 2005 7:59 am Post subject:

Hi all,

This is not a query directly about WMQI\WBIMB, but one related to the integration development process. I want to shared experiences and see if anyone had hit similar problems to us, and if so, what they have done to resolve them.

First a bit of background. Where I work we have a goal â€˜integrationâ€™ architecture that uses MQ & WMQI. We have a mixture of package applications and bespoke applications, and there is a requirement for these apps to exchange data. The goal integration architecture is designed to enable this to happen in consistent way, and in as a decoupled fashion as possible. It uses WMQI as a central message â€˜hubâ€™, and uses the publish/subscribe pattern to achieve the decoupling. The central hub is termed the â€˜Integration Layerâ€™. Source apps publish data into the Integration Layer. Target apps subscribe to these publications if required.

e.g the front office trading application (3rd party package) publishes trades (in near real time) into the Integration Layer. There are two subscribers, one is the confirmation/settlement management system (another 3rd party app) and the other is the data warehouse (a in-house build).

Source applications are supposed to publish â€˜genericâ€™ messages that conform to certain format rules i.e they are not source specific.

Ideally we want our development process to go something like this:
1. Business Analysts define the data that needs to be published from the master source application. This is done by building (or reusing) part(s) of a 'canonical data model' (along the lines of the Canonical Data Model pattern as outlined in Hohpe\Woolf book Patterns of Enterprise Application Architecture).
2. The business analysts then define a logical message to be published â€“ this is just a definition of which business elements\objects a message should contain.
3. The techies take the logical message defined in step 2, added some physical stuff e.g standard headers and footers, currency attributes for monetary amounts etc and create a XML schema to describe the published message. They then generate the XML schema, this schema is used at run-time by WMQI\WBIMB.
4. Subscribing application teams can view a catalogue of the published messages (built from step 1 & 2) and can subscribe to the relevant messages if required.
5. The Canonical data model evolves as more information is published into the Integration Layer i.e with each iteration through this development cycle.

Needless to say we are not quite there yet!

Our issue is we are looking hard for tooling to support this development process. Ideally we would have a seamless, integrated tool that allows the business analysts to do the data modelling side & building the logical messages, but also allow the techies to do the XML stuff and generate the schemas.

So far we have not seen anything suitable. Currently we have a â€˜home-grownâ€™ mix of a Notes database that holds the data model and logical message definitions and XML schemas created through XMLSpy for the documentation of the physical messages. The Notes side of things is very hard to use, so as a result we are always struggling to get the various integration teams to use it and keep it up to date. As it is not up to date, we are constantly getting duplicate date elements\objects being created, and often not detecting this until late in the development process. In addition, it is difficult to stop source application specific stuff â€˜leakingâ€™ through to the published generic messages e.g source application codes\Ids\fieldnames.

Anybody else out there having similar experiences\problems? Anybody have any suggestions\opinions on possible solns?