| Author |
Message
|
| ioat |
 Posted: Thu Dec 27, 2007 12:10 am Post subject: Could I retrieve the message not consecutively from the MQ? |
 |
|
Acolyte
Joined: 18 Jul 2005
Posts: 53
|
Say there are 10 messages in the MQ, could I retrieve the message 1,3,4,6,8,9 (they have the same prefix in the message body), and then retrieve the rest messages (they also have the same prefix)?
PS, I use the MQ C API to access the MQ 6.0 |
|
| Back to top |
|
 |
| elvis_gn |
| Posted: Thu Dec 27, 2007 12:56 am Post subject: |
|
|
Padawan
Joined: 08 Oct 2004
Posts: 1905
Location: Dubai
|
Hi ioat,
How did you come to the conclusion that you wanted the msgs in order 1,3,4....you say they all have the same prefix, then what is the determining factor ?
Regards. |
|
| Back to top |
|
|
| ioat |
| Posted: Thu Dec 27, 2007 1:52 am Post subject: |
|
|
Acolyte
Joined: 18 Jul 2005
Posts: 53
|
Thanks for your reply.
All the messages will start with a system name. Eg: some messages starts with "YOSxxxxx", and some messages starts with "YCCxxxxx"
So I want to retrieve all YOS messages, and then process them together. Then retrieve all YCC messages.
Is there such a way? |
|
| Back to top |
|
|
| elvis_gn |
| Posted: Thu Dec 27, 2007 2:34 am Post subject: |
|
|
Padawan
Joined: 08 Oct 2004
Posts: 1905
Location: Dubai
|
Hi ioat,
The way would be to browse the msg, check the tag and then do a get.....but this is a very bad solution in terms of performance...
Rather have the sender either put some sort of a priority on the msg, or identifier on the header.
Regards. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Dec 27, 2007 5:07 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
You can only directly select on MSGID or CORRELID or GROUPID, however MQ will still read the queue sequentially unless it is indexed (possible z/OS only).
So using browse and then get "under browse cursor" isn't so bad really, unless the queue is really massive and you are skipping far more than you are getting. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Dec 27, 2007 5:19 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Of course, instead of just putting the system name in the front of the message, you can also create a logical group for each system.
Then you can get by group.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| ioat |
| Posted: Thu Dec 27, 2007 5:29 am Post subject: |
|
|
Acolyte
Joined: 18 Jul 2005
Posts: 53
|
| elvis_gn wrote: |
Hi ioat,
The way would be to browse the msg, check the tag and then do a get.....but this is a very bad solution in terms of performance...
Rather have the sender either put some sort of a priority on the msg, or identifier on the header.
Regards. |
How to browse and get? If I browse the 1,2,3,4,5 messages, but I only want to process 1,3,4 together, and 2,5 together. Could I get the message not consecutively? Or I need to get all these 5 messages in memory, and then do it myself.
I use the C API, seems hard to set the header. |
|
| Back to top |
|
|
| ioat |
| Posted: Thu Dec 27, 2007 5:33 am Post subject: |
|
|
Acolyte
Joined: 18 Jul 2005
Posts: 53
|
| zpat wrote: |
You can only directly select on MSGID or CORRELID or GROUPID, however MQ will still read the queue sequentially unless it is indexed (possible z/OS only).
So using browse and then get "under browse cursor" isn't so bad really, unless the queue is really massive and you are skipping far more than you are getting. |
Do you mean I can get "under browse curor"? Yes, if it can work, that will be what I want.
But I also need a transaction, could I only browse and get messag 1,3,5 and then delete these messages together? |
|
| Back to top |
|
|
| ioat |
| Posted: Thu Dec 27, 2007 5:34 am Post subject: |
|
|
Acolyte
Joined: 18 Jul 2005
Posts: 53
|
| jefflowrey wrote: |
Of course, instead of just putting the system name in the front of the message, you can also create a logical group for each system.
Then you can get by group. |
The system name just a sample, I have many such different system names.
How to use the logical group? |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Dec 27, 2007 5:49 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
|
| Back to top |
|
|
| ioat |
| Posted: Thu Dec 27, 2007 6:01 am Post subject: |
|
|
Acolyte
Joined: 18 Jul 2005
Posts: 53
|
Thanks. But my sender program couldn't send a group of messages together. It just can send the message one by one. Also my receiver maybe not process all messages in the same group. It has a fixed buffer, and if the buffer is full, then it will just process currently recieved messages. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Dec 27, 2007 2:41 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Back up a little bit.
It seems that you have a design problem here.
You need to define what you want to process together... and this cannot be a batch... It should be an "atomic" transaction.
Your sender should put the messages constituting the atomic transaction under syncpoint and as a logical group. (see manual for grouping options and syncpoint)
You can then request to process once the logical group is complete.
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| ioat |
| Posted: Thu Dec 27, 2007 5:12 pm Post subject: |
|
|
Acolyte
Joined: 18 Jul 2005
Posts: 53
|
| fjb_saper wrote: |
Back up a little bit.
It seems that you have a design problem here.
.....
You can then request to process once the logical group is complete.
Enjoy |
I have a sender to process many system messages (as I have many many systems, I couldn't assign an individual sender for each system). And all these system messages sequence are not determinated. Then the sender will put these messages into a same queue. The receiver need to forward these messages to different place according to the system names. To get a better performance, the receiver needs to receive serveral same system massages (maybe not consecutive), and then compresse the messages before forwarding. After processing these messages, receiver needs to delete these messages from the queue.
Couldn't implement this requirement? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Dec 28, 2007 6:55 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| ioat wrote: |
I have a sender to process many system messages (as I have many many systems, I couldn't assign an individual sender for each system). And all these system messages sequence are not determinated. Then the sender will put these messages into a same queue. The receiver need to forward these messages to different place according to the system names. To get a better performance, the receiver needs to receive serveral same system massages (maybe not consecutive), and then compresse the messages before forwarding. After processing these messages, receiver needs to delete these messages from the queue.
Couldn't implement this requirement? |
This looks to me like a routing + staging pattern.
Soo:
- 1st process receives any message and routes to a local queue
1 queue per system.
- 2nd process (at least 1 instance per queue) reads multiple messages under syncpoint and puts one (compression) or multiple messages to the destination queue (syncpoint).
The other pattern you could use here without any message compressions (other than the algorythms on the channel) however would be a publish / subscribe.
Each message coming in on the input queue gets published to a topic according to a system. The consumers subscribe to the topic...
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Dec 28, 2007 7:00 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
It also looks like there's some confusion about units of work, versus browse/destructive get.
If you need to delete a message only after it's been processed, use a syncpoint. DON'T use browse/destructive get.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
