| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| A Design Issue - Process in Set/Order |
« View previous topic :: View next topic » |
| Author |
Message
|
| guogangli |
 Posted: Tue Sep 28, 2010 1:08 pm Post subject: A Design Issue - Process in Set/Order |
 |
|
Novice
Joined: 02 Mar 2009
Posts: 15
|
Experts,
We plan to convert an application to use WMQ 7.0, and we have a design issue.
Messages coming from another system are logically grouped into small sets, with about 5 messages per set. Messages in the same set can be identified by a unique Id. Messages with mixed sets are queued in a WMQ local queue.
It’s a multi-threaded application. Multiple threads retrieve messages from the queue, and process them.
The main restriction is messages in a set must be processed in certain order. To maintain the correct order, messages in the same set are processed by a one thread, synchronously. In other words, if Thread 1 starts to process messages in Set A, other threads should not process any messages in Set A, but they can process messages in other sets.
How would WMQ handle it? It seems like WMQ doesn’t provide a way to lock more than one messages.
I would appreciate very much if you have any input or suggestion.
Grant Li |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Sep 28, 2010 1:17 pm Post subject: Re: A Design Issue - Process in Set/Order |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| guogangli wrote: |
| How would WMQ handle it? |
With message groups. If the reading application specifies MQGMO_ALL_MSGS_AVAILABLE then only complete message groups will be read off. Specifying MQGMO_LOGICAL_ORDER ensures they come off in order.
Obviously this relies on the goodwill of the putting application to set the message flags.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Sep 28, 2010 2:28 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9403
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| must be processed in certain order |
Do you mean that messages in set A must be processed in the exact same order in which they were put to the queue? (Even though messages in set B may be interleaved with messages from set A, and so forth.)
Refer to the APR and APG manuals. Look for message groups. As vitor mentioned, WMQ allows for messages to be assigned a MessageGroup, which is identified by a unique GroupId in the MQMD. The qmgr assigns Groupid, just as does for MsgId.
The creating app must specify the appropriate msg flags to indicate that a message is part of a message group. The receiving app must also specify as part of the MQPUT and put message options that the next message from the queue is to be part of the group.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Tue Sep 28, 2010 3:58 pm Post subject: |
|
|
Jedi
Joined: 25 Mar 2003
Posts: 2495
Location: Melbourne, Australia
|
An similiar approach that does not use the message grouping feature of MQ:
Use correlation id to identify sets of messages. A consumer thread gets the next message, and then uses the correlation id of that message to get the others in the set. The correlation id can hint at the number of expected messages in each set, or the app can just get messages until it experiences reason code 2033.
Bear in mind that MQ does not guarantee message sequencing. One could use incremented correlation ids to define the sequence.
There will be performance considerations using these techniques if there are a large number of messages on the queue (100's - 1000's ?).
The simplest design is to combine the sets of data into one message before it is put on the queue. 
_________________
Glenn |
|
| Back to top |
|
|
| sunny_30 |
| Posted: Thu Sep 30, 2010 2:16 pm Post subject: |
|
|
Master
Joined: 03 Oct 2005
Posts: 258
|
here is an idea:
if multi threads are say total - "n"
set up "n" # of queues in MQ FIFO order
unique-ID (assuming this is an integer) modulus "n" would always result in an index 0, 1...to 4
have the app calculate modulus & route to separate queues so FIFO order among same set is always maintained
MQ-grouping each set of messages would enable you run more multi-threads on each single queue...
Last edited by sunny_30 on Thu Sep 30, 2010 2:24 pm; edited 2 times in total |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Sep 30, 2010 2:20 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9403
Location: US: west coast, almost. Otherwise, enroute.
|
A creative solution.
I'm certain that there are a wide variety of ways to skin this cat (figure of speech; no cats were injured in this post); but there are already tools in WMQ to accomplish the task, namely: message groups.
With message groups, there can be one queue to service the multiple requestors, and multiple consumers can use the same input queue.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
