| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Messages skipped by MQGET |
« View previous topic :: View next topic » |
| Author |
Message
|
| davenash |
 Posted: Mon Dec 12, 2005 8:34 am Post subject: Messages skipped by MQGET |
 |
|
Newbie
Joined: 12 Dec 2005
Posts: 7
|
Hi,
My clients are experiencing the problem of messages skipped when doing BROWSE_NEXT on an MQGET call.
I am performing the following sequence:
MQGET with MQGMO_BROWSE_NEXT, MQGMO_LOCK + MQGMO_WAIT
Then allocate a buffer depending on the message size, and
MQGET with MQGMO_NO_WAIT + MQGMO_MSG_UNDER_CURSOR + MQGMO_CONVERT + MQGMO_SYNCPOINT
Finally MQCMIT.
I found this discussion:
http://www.mqseries.net/phpBB2/viewtopic.php?t=23228&highlight=mqget+message+skipped
Which sounds like the same problem. Apparently if I am using browse, I will miss messages that are uncommitted.
Why won't they appear in the correct sequence under my browse cursor when after they have been committed?
What is the recommended way to make sure I never miss a message? Given that my second MQGET removes the message I just read in every case, should I be using BROWSE FIRST instead?
Thanks in advance
Dave contact admin |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Mon Dec 12, 2005 8:39 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
You will always skip messages that are uncommitted. They will not be available until after they are committed, and then only to the next GET that would have otherwise retrieved them.
You are probably better off using a zero length buffer, and issuing two destructive GETS. Do not use ACCEPT_TRUNCATED_MSG, and the first get will fail but give you the right size for the buffer. Then you can allocate the buffer to the size you need, and redo the get.
Or you can just allocated a 100mb or 16mb buffer everytime and do one get instead of two. 
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| davenash |
| Posted: Mon Dec 12, 2005 8:45 am Post subject: |
|
|
Newbie
Joined: 12 Dec 2005
Posts: 7
|
Hi, thanks for the quick response.
| jefflowrey wrote: |
| You will always skip messages that are uncommitted. |
I don't understand this. If they are uncommitted then as far as I am concerned they are not there at all.
| Quote: |
| They will not be available until after they are committed, and then only to the next GET that would have otherwise retrieved them. |
So why does my next GET skip them once they ARE committed?
| Quote: |
| You are probably better off using a zero length buffer, and issuing two destructive GETS. |
Not using BROWSE at all, you mean?
| Quote: |
| Do not use ACCEPT_TRUNCATED_MSG, and the first get will fail but give you the right size for the buffer. Then you can allocate the buffer to the size you need, and redo the get. |
Ah, I thought I had to use browse in order to get the same message the second time.
Why will this not skip those "uncommitted" messages when using BROWSE_NEXT will (even once committed)?
Dave |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Dec 12, 2005 8:53 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Browse uses a cursor. This cursor moves relative to the first message on the queue.
Destructive GETS do not use a cursor. A destructive get always retrieves the first message (that matches the match options). So if you have three uncommitted messages, followed by several committed message, and you do a GET, you will get the 4th message. Then, if you do another get, but the first message has been committed, you will get the first message, not the 5th.
But using BROWSE, you will get the fifth, instead of the first.
If the buffer is too small for the message, and you have not specified ACCEPT_TRUNCATED_MESSAGE, then the GET will actually fail, and not retrieve the message at all. So it will still be there for the next GET, and will likely be the first message still. To avoid the "likely" part, you can use MsgID/CorrID match options on the second GET to ensure you get the right message (assuming that this is a unique pair).
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| davenash |
| Posted: Mon Dec 12, 2005 8:57 am Post subject: |
|
|
Newbie
Joined: 12 Dec 2005
Posts: 7
|
Thanks for the explanation. I'm off to try it out
Dave |
|
| Back to top |
|
|
| davenash |
| Posted: Tue Dec 13, 2005 1:53 am Post subject: |
|
|
Newbie
Joined: 12 Dec 2005
Posts: 7
|
| mquseless wrote: |
| 1. After the first GET has failed with 2080, you do indeed get the length of the msg, and you could set the buffer to be big enough. However, there is no guarantee that when the app does its next GET it will get the same msg as previously unless no other app is reading the queue. |
Filtering on MsgID (assuming it is unique) could help, and allowing for the second GET to fail.
| Quote: |
| To future-proof the app, you should do a browse with lock, then get the msg under the cursor. A small change to the original get options wil fix the problem - change BROWSE_NEXT to BROWSE_FIRST and the app will get msgs that have been committed after the browse cursor has passed them. |
I thought this might be a solution. By "future-proof" are you referring to the possibility that more than one process could be reading the queue?
| Quote: |
| 2. Don't allocate a massive buffer just because you do not know the size of the msg; this causes out of memory problems in the agent. If you really really must do this, and it is lazy design that the app does not have a reasonable idea of the size of the msg it is expecting, set the buffer no bigger than the sallest MAXMSGL in the system. |
Don't worry, I wasn't planning on doing this. I don't think the suggestion was entirely serious (there was a smiley). However if the application uses a well-defined message size of some reasonable value (eg. 1000 bytes) then there is no reason why this couldn't be an option.
Regards
Dave |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Dec 13, 2005 4:35 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| mquseless wrote: |
| Two lots of poor advice here. |
That's really a strong term for "I disagree".
Doing one get is faster than two. Always. Doing a single get is simpler code logic than doing two gets, and will be easier to understand and maintain.
What "agent" are you talking about? The MCA? The MCA should be written in such a way to be "future-proofed" already, and have no issues at all with handling as many messages of 100MB in size as you wish to throw at it. If the MCA does have an issue with this (other than speed), then it's clearly a bug and would be addressed by PMR. Also, it's not clear that contact admin meant "applications using a client connection" by "clients'. It could have been "customers".
If you meant the application.. it's basic to ensure that an application has the runtime memory it needs to do the job. When working with a non-GC language, then one can make sure that the large message buffer is not reallocated unnecessarily. When working with a GC language, then it's not worth worrying about.
Yes, it's better to use a smaller buffer. But in some cases you should use the largest buffer possible, from simple defensive programming practices - liberal in what you accept and conservative in what you put out.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| davenash |
| Posted: Tue Dec 13, 2005 4:46 am Post subject: |
|
|
Newbie
Joined: 12 Dec 2005
Posts: 7
|
| jefflowrey wrote: |
| Also, it's not clear that contact admin meant "applications using a client connection" by "clients'. It could have been "customers". |
It was "customers" 
That ambiguity did occur to me at the time, but I (perhaps incorrectly!) thought I had expressed it clearly enough.
I have settled, assuming no more issues arise, on simply changing BROWSE_NEXT to BROWSE_FIRST (a change that someone else also suggested to me) for several reasons. Firstly it is the simplest change. And also it avoids the "multiple readers" problem.
Dave |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
