Author |
Message
|
ioat |
Posted: Thu Dec 27, 2007 12:10 am Post subject: Could I retrieve the message not consecutively from the MQ? |
|
|
Acolyte
Joined: 18 Jul 2005 Posts: 53
|
Say there are 10 messages in the MQ, could I retrieve the message 1,3,4,6,8,9 (they have the same prefix in the message body), and then retrieve the rest messages (they also have the same prefix)?
PS, I use the MQ C API to access the MQ 6.0 |
|
Back to top |
|
 |
elvis_gn |
Posted: Thu Dec 27, 2007 12:56 am Post subject: |
|
|
 Padawan
Joined: 08 Oct 2004 Posts: 1905 Location: Dubai
|
Hi ioat,
How did you come to the conclusion that you wanted the msgs in order 1,3,4....you say they all have the same prefix, then what is the determining factor ?
Regards. |
|
Back to top |
|
 |
ioat |
Posted: Thu Dec 27, 2007 1:52 am Post subject: |
|
|
Acolyte
Joined: 18 Jul 2005 Posts: 53
|
Thanks for your reply.
All the messages will start with a system name. Eg: some messages starts with "YOSxxxxx", and some messages starts with "YCCxxxxx"
So I want to retrieve all YOS messages, and then process them together. Then retrieve all YCC messages.
Is there such a way? |
|
Back to top |
|
 |
elvis_gn |
Posted: Thu Dec 27, 2007 2:34 am Post subject: |
|
|
 Padawan
Joined: 08 Oct 2004 Posts: 1905 Location: Dubai
|
Hi ioat,
The way would be to browse the msg, check the tag and then do a get.....but this is a very bad solution in terms of performance...
Rather have the sender either put some sort of a priority on the msg, or identifier on the header.
Regards. |
|
Back to top |
|
 |
zpat |
Posted: Thu Dec 27, 2007 5:07 am Post subject: |
|
|
 Jedi Council
Joined: 19 May 2001 Posts: 5866 Location: UK
|
You can only directly select on MSGID or CORRELID or GROUPID, however MQ will still read the queue sequentially unless it is indexed (possible z/OS only).
So using browse and then get "under browse cursor" isn't so bad really, unless the queue is really massive and you are skipping far more than you are getting. |
|
Back to top |
|
 |
jefflowrey |
Posted: Thu Dec 27, 2007 5:19 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002 Posts: 19981
|
Of course, instead of just putting the system name in the front of the message, you can also create a logical group for each system.
Then you can get by group. _________________ I am *not* the model of the modern major general. |
|
Back to top |
|
 |
ioat |
Posted: Thu Dec 27, 2007 5:29 am Post subject: |
|
|
Acolyte
Joined: 18 Jul 2005 Posts: 53
|
elvis_gn wrote: |
Hi ioat,
The way would be to browse the msg, check the tag and then do a get.....but this is a very bad solution in terms of performance...
Rather have the sender either put some sort of a priority on the msg, or identifier on the header.
Regards. |
How to browse and get? If I browse the 1,2,3,4,5 messages, but I only want to process 1,3,4 together, and 2,5 together. Could I get the message not consecutively? Or I need to get all these 5 messages in memory, and then do it myself.
I use the C API, seems hard to set the header. |
|
Back to top |
|
 |
ioat |
Posted: Thu Dec 27, 2007 5:33 am Post subject: |
|
|
Acolyte
Joined: 18 Jul 2005 Posts: 53
|
zpat wrote: |
You can only directly select on MSGID or CORRELID or GROUPID, however MQ will still read the queue sequentially unless it is indexed (possible z/OS only).
So using browse and then get "under browse cursor" isn't so bad really, unless the queue is really massive and you are skipping far more than you are getting. |
Do you mean I can get "under browse curor"? Yes, if it can work, that will be what I want.
But I also need a transaction, could I only browse and get messag 1,3,5 and then delete these messages together? |
|
Back to top |
|
 |
ioat |
Posted: Thu Dec 27, 2007 5:34 am Post subject: |
|
|
Acolyte
Joined: 18 Jul 2005 Posts: 53
|
jefflowrey wrote: |
Of course, instead of just putting the system name in the front of the message, you can also create a logical group for each system.
Then you can get by group. |
The system name just a sample, I have many such different system names.
How to use the logical group? |
|
Back to top |
|
 |
jefflowrey |
Posted: Thu Dec 27, 2007 5:49 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002 Posts: 19981
|
|
Back to top |
|
 |
ioat |
Posted: Thu Dec 27, 2007 6:01 am Post subject: |
|
|
Acolyte
Joined: 18 Jul 2005 Posts: 53
|
Thanks. But my sender program couldn't send a group of messages together. It just can send the message one by one. Also my receiver maybe not process all messages in the same group. It has a fixed buffer, and if the buffer is full, then it will just process currently recieved messages. |
|
Back to top |
|
 |
fjb_saper |
Posted: Thu Dec 27, 2007 2:41 pm Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
Back up a little bit.
It seems that you have a design problem here.
You need to define what you want to process together... and this cannot be a batch... It should be an "atomic" transaction.
Your sender should put the messages constituting the atomic transaction under syncpoint and as a logical group. (see manual for grouping options and syncpoint)
You can then request to process once the logical group is complete.
Enjoy  _________________ MQ & Broker admin |
|
Back to top |
|
 |
ioat |
Posted: Thu Dec 27, 2007 5:12 pm Post subject: |
|
|
Acolyte
Joined: 18 Jul 2005 Posts: 53
|
fjb_saper wrote: |
Back up a little bit.
It seems that you have a design problem here.
.....
You can then request to process once the logical group is complete.
Enjoy  |
I have a sender to process many system messages (as I have many many systems, I couldn't assign an individual sender for each system). And all these system messages sequence are not determinated. Then the sender will put these messages into a same queue. The receiver need to forward these messages to different place according to the system names. To get a better performance, the receiver needs to receive serveral same system massages (maybe not consecutive), and then compresse the messages before forwarding. After processing these messages, receiver needs to delete these messages from the queue.
Couldn't implement this requirement? |
|
Back to top |
|
 |
fjb_saper |
Posted: Fri Dec 28, 2007 6:55 am Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
ioat wrote: |
I have a sender to process many system messages (as I have many many systems, I couldn't assign an individual sender for each system). And all these system messages sequence are not determinated. Then the sender will put these messages into a same queue. The receiver need to forward these messages to different place according to the system names. To get a better performance, the receiver needs to receive serveral same system massages (maybe not consecutive), and then compresse the messages before forwarding. After processing these messages, receiver needs to delete these messages from the queue.
Couldn't implement this requirement? |
This looks to me like a routing + staging pattern.
Soo:
- 1st process receives any message and routes to a local queue
1 queue per system.
- 2nd process (at least 1 instance per queue) reads multiple messages under syncpoint and puts one (compression) or multiple messages to the destination queue (syncpoint).
The other pattern you could use here without any message compressions (other than the algorythms on the channel) however would be a publish / subscribe.
Each message coming in on the input queue gets published to a topic according to a system. The consumers subscribe to the topic...
Enjoy  _________________ MQ & Broker admin |
|
Back to top |
|
 |
jefflowrey |
Posted: Fri Dec 28, 2007 7:00 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002 Posts: 19981
|
It also looks like there's some confusion about units of work, versus browse/destructive get.
If you need to delete a message only after it's been processed, use a syncpoint. DON'T use browse/destructive get. _________________ I am *not* the model of the modern major general. |
|
Back to top |
|
 |
|