| Author |
Message
|
| PeterPotkay |
 Posted: Fri Feb 17, 2006 11:29 am Post subject: JMS Always Browses? |
 |
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Using Transaction Vision, an API Exit, I observed th efollowing behaviour for a JMS app draining a queue.
The app issues a MQ Browse for the first message on the queue, with a 4 K buffer length. This fails with a warning, that says the message is bigger than 4K.
The app issues a second browse for the first message on the queue, with the larger buffer size that it learned from the first browse. That works.
The app then issue a destructive get for the specific message referenced by the 2nd browse. I can see this because unlike the 1st 2 calls, this calls specifies a particular Correl ID (the one it got from the first 2 calls). It also uses the Get Message Under Browse cursor.
And then it just keeps repeating this. The 3 calls take about 1/100th of a second to complete, and then the app does "app stuff" for a 1/10 of a second before going after the next message with another round of 3 MQGETs. Message are persistent, and about 4.5 K in length.
We want to increase throughput, which is now only about 600 destructive MQGETs per minute. This is all MQ Client residing on separate servers from the QM.
Sooooo,
Can a JMS app issue a straight MQGET call that does not include a "preview" browse?
Can the buffer that is allocated for any MQGET by JMS be set by the app?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Fri Feb 17, 2006 12:28 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Is this with MDBs or with a JMS Receiver?
I think, yes, the MDB Listener always browses looking at each message - and I doubt the JMS Receiver is any better.
In particular, it has to do this in order to support JMS Selectors - even by message ID.
If you really want to increase performance, don't use JMS...
If you're using MDBs, you can also look at increasing the threads that the MDB Listener uses.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| mvic |
| Posted: Fri Feb 17, 2006 3:02 pm Post subject: Re: JMS Always Browses? |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| PeterPotkay wrote: |
| Can a JMS app issue a straight MQGET call that does not include a "preview" browse? |
Sorry to answer a question with a question - but is a selector being used? If not, I would expect a straight MQGET to be called. It sounds like a selector is being used in the case you describe, but having said that, I am surprised by the second MQGET/BROWSE - what precisely are the MQGMO flags?
| Quote: |
| Can the buffer that is allocated for any MQGET by JMS be set by the app? |
JMS doesn't provide for this. One of the benefits / consequences of using JMS is that the specification is fixed and published. This is a benefit because people can hope to have an app that is compatible between vendors. Another consequence of this is a lack of ... ummm ... tuning opportunities.  |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Feb 17, 2006 4:00 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| jefflowrey wrote: |
| Is this with MDBs or with a JMS Receiver? |
Waiting for the apps response, I suspect MDB.
| mvic wrote: |
| Sorry to answer a question with a question - but is a selector being used? |
I have to say no, based on the API calls I see being executed. See below.
| mvic wrote: |
| what precisely are the MQGMO flags? |
First MQGET:
No Correl ID / Message ID specified
Buffer 4096
MQGMO_NO_SYNCPOINT|MQGMO_BROWSE_FIRST|MQGMO_FAIL_IF_QUIESCING 0x2014
MQMO_MATCH_MSG_ID|MQMO_MATCH_CORREL_ID 0x3
Returns MQRC_TRUNCATED_MSG_FAILED 2080
Second MQGET:
No Correl ID / Message ID specified
Buffer set to value returned from MQGET #1
MQGMO_NO_SYNCPOINT|MQGMO_BROWSE_FIRST|MQGMO_FAIL_IF_QUIESCING 0x2014
MQMO_MATCH_MSG_ID|MQMO_MATCH_CORREL_ID 0x3
Returns MQRC_NONE 0
Third MQGET
Both Message ID AND Coorel ID are specified (the ones returned on MQGET #2)
Buffer set to value returned from MQGET #1
MQGMO_NO_SYNCPOINT|MQGMO_MSG_UNDER_CURSOR|MQGMO_FAIL_IF_QUIESCING 0x2104
MQMO_MATCH_MSG_ID|MQMO_MATCH_CORREL_ID 0x3
Returns MQRC_NONE 0
Message finally pulled off of the queue!
Based on this, I would say they are not using selectors, yet JMS is a pig.
Any hope?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Feb 17, 2006 6:30 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| PeterPotkay wrote: |
| Based on this, I would say they are not using selectors, yet JMS is a pig. |
The one time I've managed to get to T&m was just as JMS support was coming out. They warned then that it was at least 30% slower than the Java MQ Api - because the App Server had to be involved...
| PeterPotkay wrote: |
| Any hope? |
More instances of the MDB listener.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Feb 17, 2006 7:12 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Are you sure they are not doing a JMS browse in the app and then getting the message from the browse by message id ?
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mvic |
| Posted: Sat Feb 18, 2006 12:47 pm Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| PeterPotkay wrote: |
First MQGET:
No Correl ID / Message ID specified
Buffer 4096
MQGMO_NO_SYNCPOINT|MQGMO_BROWSE_FIRST|MQGMO_FAIL_IF_QUIESCING 0x2014
MQMO_MATCH_MSG_ID|MQMO_MATCH_CORREL_ID 0x3
Returns MQRC_TRUNCATED_MSG_FAILED 2080
Second MQGET:
No Correl ID / Message ID specified
Buffer set to value returned from MQGET #1
MQGMO_NO_SYNCPOINT|MQGMO_BROWSE_FIRST|MQGMO_FAIL_IF_QUIESCING 0x2014
MQMO_MATCH_MSG_ID|MQMO_MATCH_CORREL_ID 0x3
Returns MQRC_NONE 0
Third MQGET
Both Message ID AND Coorel ID are specified (the ones returned on MQGET #2)
Buffer set to value returned from MQGET #1
MQGMO_NO_SYNCPOINT|MQGMO_MSG_UNDER_CURSOR|MQGMO_FAIL_IF_QUIESCING 0x2104
MQMO_MATCH_MSG_ID|MQMO_MATCH_CORREL_ID 0x3
Returns MQRC_NONE 0
Message finally pulled off of the queue!
Based on this, I would say they are not using selectors, yet JMS is a pig. |
I'd say the above is consistent with the use of a selector - (1) first browsing to see the contents of the message, the default buffer of 4 Kb is too small, so (2) the browse is repeated with a bigger buffer so that the whole message can be read, and if the message matches the selector, (3) removing the message with get-by-correlId and get-by-msgId.
EDIT: remove my 2nd paragraph - the match options seem OK to me now |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sat Feb 18, 2006 1:09 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| mvic wrote: |
I'd say the above is consistent with the use of a selector - (1) first browsing to see the contents of the message, the default buffer of 4 Kb is too small, so (2) the browse is repeated with a bigger buffer so that the whole message can be read, and if the message matches the selector, (3) removing the message with get-by-correlId and get-by-msgId.
|
But its always the first message on the queue that matchs their selector? That doesn't make sense.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Sat Feb 18, 2006 1:36 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| PeterPotkay wrote: |
| But its always the first message on the queue that matchs their selector? That doesn't make sense. |
I think it would be really hard to code the MDB Listener so it knew what selectors were going to be used at runtime.
I think the MDB listener always browses every message, builds an index, and uses that to apply selectors. Even if no selectors are used.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| mvic |
| Posted: Sat Feb 18, 2006 2:28 pm Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| PeterPotkay wrote: |
| But its always the first message on the queue that matchs their selector? That doesn't make sense. |
It depends what the selector actually is, I suppose. Do you have the source, to check this out? (Or perhaps an MQJMS trace would give details of the selector in use?)
To the best of my knowledge, Browse is used for selectors, and (for pubsub only) noLocal condition checking. If you can prove that 3 MQGETs are being executed in the case where neither of the above apply, then I am surprised. And I think that would be a problem worth addressing with Support. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Feb 20, 2006 11:42 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Turns out they are using a JMS Receiver.
And, they were using a selector, just the Correl ID. Still couldn't figure out why then was the first browse always matching for them. They loaded up the queue with 5000 unique messages, each with a *sequential* Correl ID. Then when the GETs were run, they were asking for a specific Correl ID, but incrementing the value in the Correl ID by one, so the next message in the queue always had the next Correl ID.
I asked them to change the code to prepend "ID:" in front of the requested Correl ID. That should eliminate JMS doing the message selection via browsing, and instead ask the QM to do a true GET by Correl ID. That should eliminate 1 of the 2 browses described above.
It sounds like there is no way to bump up the default buffersize that JMS asks for over the value of 4K, so looks like we are stuck with that MQGET that will fail with a truncated messages.
Once they rerun the test, I'll see if that did get rid of one of the browses and turn the browse into a detructive get right of the bat and report back.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| wschutz |
| Posted: Mon Feb 20, 2006 12:36 pm Post subject: |
|
|
Jedi Knight
Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)
|
| Quote: |
| Using Transaction Vision, an API Exit, |
Peter, is that literaly true? Last time I looked at TV, it was an entire library (dll, so) replacement, not the API Exit (except on CICS).,
_________________
-wayne |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Feb 20, 2006 12:38 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Yeah, its an API Exit on the distributed platforms.
Used to be a replacment amqrmppa.exe. But the exit is much cleaner.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mvic |
| Posted: Mon Feb 20, 2006 12:49 pm Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| PeterPotkay wrote: |
| Still couldn't figure out why then was the first browse always matching for them. |
Did I read you right earlier, when you said no correlId or msgId were specified in the first two MQGET calls - did you mean they were MQCI_NONE and MQMI_NONE - ie. 24 bytes of 0x00 ?
I looked this up in the APR the other day... If MQCI_NONE is in md.CorrelId, it's as if no match condition is specified for correlId. If MQMI_NONE is in md.MsgId, it's as if no match condition is specified for msgId.
So perhaps this explains why every MQGET returns a message, even though the match options are apparently switched 'on'. By specifying MQCI_NONE and MQMI_NONE, they are effectively 'off'. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Feb 20, 2006 4:19 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
mvic, what happens when a JMS app does an MQGET with any type of selector is that JMS simply starts asking the QM for a copy of the next message on the queue. The JMS code then determines if the message is a match. If it is, it then issues a destructive MQGET for the message under the browse cursor. Extremely inefficient, especially whe dealing with deep queues.
This is the only way when the selector is not the Message ID and/or the Correl ID exclusivly, since MQ does not allow any other selective criteria for MQGETs.
The wrinkle is that when the JMS app chooses to use only the Correl ID and/or the Message ID as a selector, you would think the JMS spec would have passed along a destructive MQGET asking the QM for that particular message. We all know MQ can do that. No need to browse 1000 messages if message 1001 has the Correl ID you need, the QM will do it for you.
For whatever reason, you need to prepend the Message ID and or Correl ID with "ID:" (no quotes). This forces the JMS app to do a pure selective MQGET, instead of all these browses.
In this test scenario, I didn't realize this is what was happening, since I didn't see tons of browses for every selective GET. Because they app had prepopulated the queue with sequential Correl IDs, and then did gets with sequential correl ID, they kept getting one message after another, but they were getting the browse behaviour because they were not prepending the ID:.
Tomorrow they will try again, this time using ID: in front of their Correl ID on the get. I'm also asking they mix up the messages a bit on the queue, so that they are not all lined up in the exact order they want. Otherwise I don't think that is the best test of selective gets, even though they technically are.
Question remains though (I think the answer is no) on is there a way to up the 4K buffer if you know ahead of time the messages are bigger than 4K, to avoid the browse or selective destructive get that's gonna fail with MQRC_TRUNCATED_MSG_FAILED.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
|
|
