| Author |
Message
|
| Glass |
 Posted: Thu Sep 20, 2007 10:23 am Post subject: Sending multiple messages grouped by GroupId as one message |
 |
|
Acolyte
Joined: 02 Mar 2006
Posts: 56
|
Hi,
I am using Broker 6.0 and I need some help/advise on an issue that we have. We will be getting multiple messages that are grouped by GroupId in the input queue. We need to have a single XML file put in the output queue for all the messages with the same GroupId as one message in the queue.
For example, lets say we get 100 messages in the queue and the first 50 is GroupId 1 and the last 50 messages are GroupId 2. We need to have 2 messages in the output queue, one with GroupId 1 (which contains data for all 50 GroupId 1) and the other with GroupId 2 (which contains data for all 50 GroupId 2).
Has anybody run into this situation and what are your suggestions to tackle this?
Thanks! |
|
| Back to top |
|
 |
| santy |
| Posted: Fri Sep 21, 2007 6:40 am Post subject: Sending multiple messages grouped by GroupId as one message |
|
|
Centurion
Joined: 03 Nov 2006
Posts: 141
|
Hi Glass,
1) If you have field in the input message as GroupID then try to code the logic in compute node to group all the similar message.
2)See the "Propagate" statement.
3)See the "Logical order" property fo MQInput Node. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Sep 21, 2007 6:50 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Santy - That's not going to solve the problem.
The problem, as such, is that the MQInput node is only every going to send out one message in the group at a time.
A complicating factor on this is that the MQGet node can't read from the same queue that the MQInput node is reading from.
Glass - you will need to use some form of Aggregation for this.
Another complicating factor is that the Aggregation nodes want to aggregate requests from ONE input message, rather than from MANY input messages.
It may be best, with the current situation, to use perhaps the Warehouse node to save the messages into a database, and use the Last Message In Group Flag to decide when you want to start Retrieving messages from the database.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| Glass |
| Posted: Fri Sep 21, 2007 12:45 pm Post subject: |
|
|
Acolyte
Joined: 02 Mar 2006
Posts: 56
|
Thanks guys. I think we will try the db store thing and read off the db after the last message is stored process.
By the way, is MQGS_LAST_MSG_IN_GROUP a char of either 'Y' or 'N'? |
|
| Back to top |
|
|
| TonyD |
| Posted: Sun Sep 23, 2007 9:37 pm Post subject: |
|
|
Knight
Joined: 15 May 2001
Posts: 540
Location: New Zealand
|
| I spent a bit of time investigating this a while ago. It is not too difficult to get working. I stored the incoming messages in a SHARED ROW and then checked whether MQMD.MsgFlags = 24 (MESSAGE_IN_GROUP + LAST_MESSAGE_IN_GROUP); if true then carried on and built the single output message. Database would be OK also of course. You might have to have multiple flow instances if you are likely to have several concurrent groups in flight. |
|
| Back to top |
|
|
| Glass |
| Posted: Mon Sep 24, 2007 6:11 am Post subject: |
|
|
Acolyte
Joined: 02 Mar 2006
Posts: 56
|
What is a SHARED ROW and how do I store it there?
Thanks! |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Sep 24, 2007 9:52 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
ROW is a new ESQL datatype in v6. A ROW variable is just another tree - so it allows you to create your own version of Environment or LocalEnvironment, without using those trees themselves.
SHARED allows an ESQL variable to exist across all instances of a given message flow, and can be scoped either to exist in ALL Compute nodes in the flow, or only in all instances of ONE Compute node in a flow.
So a SHARED ROW is a tree like Environment, except it exists and persists across all instances of a message flow.
You can see examples of using and creating them in the Sample for Message Routing.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| Glass |
| Posted: Mon Sep 24, 2007 11:33 am Post subject: |
|
|
Acolyte
Joined: 02 Mar 2006
Posts: 56
|
|
| Back to top |
|
|
| EddieA |
| Posted: Tue Sep 25, 2007 5:50 pm Post subject: |
|
|
Jedi
Joined: 28 Jun 2001
Posts: 2453
Location: Los Angeles
|
Remember to give some thought to how this might work, if you have some kind of error part way through the Group. The messages up to that point have been read, and removed from the queue.
Cheers,
_________________
Eddie Atherton
IBM Certified Solution Developer - WebSphere Message Broker V6.1
IBM Certified Solution Developer - WebSphere Message Broker V7.0 |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Sep 25, 2007 6:42 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Store to DB will allow for better scalability...
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| 4integration |
| Posted: Wed Jul 02, 2008 10:19 pm Post subject: |
|
|
Disciple
Joined: 04 Sep 2006
Posts: 197
Location: Gothenburg, Sweden
|
Hello,
It would be really nice if IBM (or other) could publish some patterns for scenarios such this. I think it would be of common interest.
I have similar problem as have been posted here: http://www.mqseries.net/phpBB2/viewtopic.php?p=219125
_________________
Best regards
4 Integration |
|
| Back to top |
|
|
| billybong |
| Posted: Thu Jul 03, 2008 2:56 am Post subject: |
|
|
Disciple
Joined: 22 Jul 2005
Posts: 150
Location: Stockholm, Sweden
|
The collection node in 6.1 might be exactly what you're after.
I haven't worked with it myself, but it seems it could fit your needs.
http://www.ibm.com/developerworks/websphere/library/techarticles/0805_coleman/0805_coleman.html
Again, that requires you to decide whether an upgrade is worth this feature.
_________________
IBM Certified Solution Developer - WebSphere Message Broker V6.1
IBM Certified Solution Developer - WebSphere Integration Developer V6.0
IBM Certified System Administrator - WebSphere MQ V6.0
IBM Certified Solution Developer - WebSphere DataPower |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Thu Jul 03, 2008 12:52 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
The Collector node in V6.1 might do the job IF the number of messages in the group was :-
1) Small (say <10)
2) Constant
A more scalable way is as already been mentioned is to store the messages in a DB and when the last message in the group is received, start the processing for the whole group.
This method will also work for multiple instances of the flow.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| wyatt |
| Posted: Thu Jul 03, 2008 6:46 pm Post subject: |
|
|
Voyager
Joined: 28 Nov 2004
Posts: 76
|
| Is the SHARED ROW available in the java compute node? |
|
| Back to top |
|
|
|
|
