ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » WBIMB Pub/Sub when subscribers need custom transformation?

Post new topic  Reply to topic
 WBIMB Pub/Sub when subscribers need custom transformation? « View previous topic :: View next topic » 
Author Message
Sandman
PostPosted: Mon Apr 10, 2006 8:50 am    Post subject: WBIMB Pub/Sub when subscribers need custom transformation? Reply with quote

Centurion

Joined: 16 Oct 2001
Posts: 134
Location: Lincoln, RI

Hi,

I'm hoping you can help me understand this. I've been away from WBIMB the past couple years, but there are still others in our company who still use the product more regularly. I have searched this forum and read a lot about Pub/Sub, and I've also been through the manuals.

Here's our situation:

Though some subscribers can digest our "enterprise" (canonical) message format, others require subscriptions in specific formats (some might not even be XML, i.e. COBOL).

One consideration is the use of Subscription Points. An alternative is an "adapter" approach, attempting to provide looser coupling of Publisher/Subscriber.

Note: Regardless of which approach we take, we are considering creating our own Subscription XML schema to "wrap" the sub commands that an MQ app. can store in the RFH2/psc folder, and having a "Subscription" message flow parse this XML message to perform the actual subscription registrations.

In the Subcription Points approach, in our situation, we'd need to have a separate path through the flow that is processing an external system's "publication" for each unique subscription format. That way, each path would format appropriately, then create a unique Publication Node for its unique subscription format. Then we'd just have any subscribers who require that format specify a subscription point that equates to "their" pub. node.

If this is correct, then would our topics end up something like "ModifyPartyFormat1", "ModifyPartyFormat2", etc.?

In the Adapter approach, the goal is to provide more flexibility (i.e. if/when a non-canonical-compliant subscriber someday gets "on board" w/ our enterprise format, or we replace that component w/ something that does, we could more easily remove the adapter(s)).

For each subscriber app that needs its message format adapted (and can't/won't do it itself), create a pair of queues and message flows (on the broker). For example, for App1, who wants its subscription delivered to Q1, create queues called App1_Sub_Adapter_Q and App1_Pub_Adapter_Q and message flows called App1_Sub_Adapter and App1_Pub_Adapter.

App1 would then send a subscription message to its subscription adapter queue, using the same "ModifyParty" topic that all other subscribers to this event would use (though they would send theirs to the "master" Subscription_Q), and specifying their subscriber queue. The App1_Sub_Adapter message flow would first cache (in a database table) the actual subscriber queue and qmgr names, along w/ the name of its pub adapter queue. Then it would change the subscriber queue in the message from the actual to the pub adapter queue, and send the message to the Subscription_Q, where the Subscription message flow would register the subscription, but to the pub adapter queue.

Now the message flow that does the publishing would only publish "ModifyParty" events, which the broker would detect, and in turn, send a subscription message to the App1_Pub_Adapter_Q queue (and also to any other subscriber queues for apps which did not require custom formats).

The App1_Pub_Adapter message flow - whose sole purpose is to specifically adapt subscriptions for this particular app - would perform custom transformation, then reference the cache table to convert the input queue name on which the message arrived to the actual queue name sent originally by the subscriber (then of course, send the message to it).

Would greatly appreciate input on pros/cons of these 2 approaches, as well as any other approaches.

Thank you,
Mike
Back to top
View user's profile Send private message Send e-mail
RocknRambo
PostPosted: Mon Apr 10, 2006 1:37 pm    Post subject: Reply with quote

Partisan

Joined: 24 Sep 2003
Posts: 355

our requirements are something similar.. and its close to the approach#1 specified..but not exactly

We define our Enterprise Messages common canonical format for the messages flowing across the hub and leverage Pub-Sub Model by defining the Topics in Enterprise Standard.
Every app (source/Target) are totally isolated.


App1 send its own format to the Hub,

First Msg Flow wud transform to the Enterprise message and publish with a topic .

Now, this Message is avail Enterprise Level ... lets say, there 3 Apps interested in tht message.

App2 (understands enterprise format)
App3 (understands flat file)
App4 (understands CwXML) - IBM WBI Adapter.

1 queue will subscribe where App2 listens on it.
2 Message flows (their input queues) will subscribe to the Topic in the Enterprise Format and transform to the Target format.

hope this wud help in startup.
-RR
Back to top
View user's profile Send private message
jefflowrey
PostPosted: Mon Apr 10, 2006 2:53 pm    Post subject: Reply with quote

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

Another option, rather than screwing around with a database to store format-specific destinations, is to add another topic.

So if the original message actually wants to subscribe to topic "a/b/c", but needs format 123, then your meta-subscriber would create two subscriptions - one for "a/b/c/" to the message flow that will reformat the data into format 123, and another that is "a/b/c/123" for the original subscription request.

Then the reformatting message flow receives messages for topic "a/b/c", and republishes them as "a/b/c/123".

This is close to the subscription point mechanism.
_________________
I am *not* the model of the modern major general.
Back to top
View user's profile Send private message
Sandman
PostPosted: Wed Apr 12, 2006 5:05 am    Post subject: Reply with quote

Centurion

Joined: 16 Oct 2001
Posts: 134
Location: Lincoln, RI

Thanks for the replies.

Jeff, I like the double-subscription idea in lieu of using a database.

What are your thoughts/experience w/ subscription points? At first glance, they seem like they more tightly couple subscribers to publishers, as the subs are specifically subscribing to particular pub nodes? Is it better to get granular using deeper topic hierarchies, as in your double-subscription technique? Do sub points offer some other benefit? Or are they interchangeable?

Thanks again,
Mike
Back to top
View user's profile Send private message Send e-mail
jefflowrey
PostPosted: Wed Apr 12, 2006 5:39 am    Post subject: Reply with quote

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

I don't have a lot of hands-on with subscription points. I remembered, apparently incorrectly, that different subscription points ended up requiring different publication queues or something - but that's not the case.

I'd say if you're worried about performance then you should check the performance reports to see what they say about pub/sub and if they make comments about deep topic hierarchies versus flat hierarchies. You can also see if they make comments about subscription points versus topics - but I doubt they will.

Another way to slice the problem using topics is to use different topic roots - if you have a fixed set of formats. So it would be "123/a/b/c" instead of "a/b/c/123". This could help limit your topic depth if that ends up being a performance factor.

Performance aside, my instinct is that they should be equivalent - but there's some reason for allowing subscription points in the first place. Maybe it's just the whim of an architect who thought topic hierarchies should be clean of messy details such as "formatting", or maybe there's a deep reason.

As always - it's better to test it yourself if you can.
_________________
I am *not* the model of the modern major general.
Back to top
View user's profile Send private message
Sandman
PostPosted: Wed Apr 12, 2006 5:44 am    Post subject: Reply with quote

Centurion

Joined: 16 Oct 2001
Posts: 134
Location: Lincoln, RI

Ok, I'll poke around some more Jeff. BTW, do you happen to have a handy link to the performance reports you mentioned?

Thank you.
Back to top
View user's profile Send private message Send e-mail
jefflowrey
PostPosted: Wed Apr 12, 2006 6:16 am    Post subject: Reply with quote

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

I keep a handy link availble to the main support pack page...
http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27007197#1

It should have the performance reports for Broker in there somewhere.
_________________
I am *not* the model of the modern major general.
Back to top
View user's profile Send private message
RocknRambo
PostPosted: Wed Apr 12, 2006 7:50 am    Post subject: Reply with quote

Partisan

Joined: 24 Sep 2003
Posts: 355

For the performance, we did face issues when the message size is huge around 15 - 20 Meg in Pub-Sub Model.

so, we re-designed the particular message flow to use DestinationList. Or we had to Isloate the Publishing flow into differnt EG.

Jeff,

any suggestions on this..

the way we do publishing is..
form the topic in RFH2 header and bundle the inputmessage as BLOB.
I'm not sure if there is a better way..

any thoughts.

-RR
Back to top
View user's profile Send private message
jefflowrey
PostPosted: Wed Apr 12, 2006 7:56 am    Post subject: Reply with quote

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

How the input message is bundled before it goes to the publication node shouldn't really matter a lot - you have to do the same amount of work to marshall the logical tree into a bitstream no matter if it's done at the publication node or earlier in the flow.

Using larger messages will always have an impact on performance - because it will always take more time to process the data.

I haven't read the Performance report to see what it says on Pub/Sub. I don't have extensive hands-on experience with Pub/Sub. So I don't really have any more suggestions.
_________________
I am *not* the model of the modern major general.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » WBIMB Pub/Sub when subscribers need custom transformation?
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.