| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| JMS Selective Gets |
« View previous topic :: View next topic » |
| Author |
Message
|
| PeterPotkay |
 Posted: Wed Apr 13, 2005 1:47 pm Post subject: JMS Selective Gets |
 |
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
It is my understanding that a JMS client that does a selective get by a value inside the message data buffer, or a MQMD field other than the Correl ID or the Message ID, will perform poorly on deep queues because the QM will send the JMS Client a complete copy of each message one by one until the match is made. The JMS client is looking at a copy of each message on the client side one by one to see if it meets the criteria.
Is this correct?
If yes, would the same JMS client perform these gets better if it did the selection by CorrelID. In that case, would something special happen on the QM side, and only one message would be returned down the MQI channel, the one the QM was able to determine matched the Correl ID?
Or do selective JMS gets always send a copy (even for Corre lID or Message ID matches) of the entire message for the JMS client to decide if it is a match or not?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Thu Apr 14, 2005 5:06 am Post subject: Re: JMS Selective Gets |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| PeterPotkay wrote: |
It is my understanding that a JMS client that does a selective get by a value inside the message data buffer, or a MQMD field other than the Correl ID or the Message ID, will perform poorly on deep queues because the QM will send the JMS Client a complete copy of each message one by one until the match is made. The JMS client is looking at a copy of each message on the client side one by one to see if it meets the criteria.
Is this correct? |
That's my understanding.
| PeterPotkay wrote: |
| If yes, would the same JMS client perform these gets better if it did the selection by CorrelID. In that case, would something special happen on the QM side, and only one message would be returned down the MQI channel, the one the QM was able to determine matched the Correl ID? |
I believe that this is what happens. Although it may depend entirely on the JMS provider.
| PeterPotkay wrote: |
| Or do selective JMS gets always send a copy (even for Corre lID or Message ID matches) of the entire message for the JMS client to decide if it is a match or not? |
Again, I don't think that the WAS WebSphere MQ JMS Provider will act this way.
BUT there are very very good reasons not to use Correlation ID as a user data filter field. It's very small, no character set translation occurs, it's binary data not string data, and it might be used for other things in the network that a particular application doesn't know about.
ALSO, using Correlation ID like this is very much trying to use a queue as a database. This is an anti-pattern in MQ design. If you need to sort messages by some piece of data, then build a simple sorter/filter and use different queues for different pieces of data. If you don't need to sort messages, then the program should just use FIFO and leave it at that.
Then your network is explicit about where each type of data is, and problem identification granularity is maintained and exposed both to the developers and to the admins ("oh, there are messages sitting on X.CUSTOMER.DATA, so the problem must be with the CUSTOMER program") .
Any time developers ask admins about this type of information, the admins should work with the developers to find a different way to accomplish the task they are really trying to do (which is likely not get messages by correlation ID). It is really better for everyone and the business.
In my opinion.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Apr 14, 2005 5:38 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Jeff,
The app is going to be run by app to 150 concurrent users. The reason for the selective GETs is the common one. We were going to use 1 queue, and just let all the apps put and do selective gets from the same queue.
Sounds like if we stay with the one queue design, we would have to test the Correl ID JMS gets versus some other selector, to see if it performs better.
This of course will now turn into a discussion of whether we should just use perm dyn queues. The apps could create dynamic queues based on the computer name running the app. And then each instance does the unqualified gets from its own queue. I just worry about uncontrolled creation of dynamic queues. From an MQ admin perspective, the one queue seems safer.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Apr 14, 2005 5:59 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| PeterPotkay wrote: |
| I just worry about uncontrolled creation of dynamic queues. |
This is a good worry. dynamic queues use resources. Uncontrolled queue creation can run a box out of needed resources pretty quick, and that's bad for everyone.
| PeterPotkay wrote: |
| From an MQ admin perspective, the one queue seems safer. |
But again, how do you know which user is failing to get messages, if the queue is filling up?
There was a somewhat in-depth discussion of a similar configuration here a few weeks ago or so.
And in fact, it contains information that we've both apparently forgotten...
http://www.mqseries.net/phpBB2/viewtopic.php?t=20026&postdays=0&postorder=asc&start=0
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Apr 14, 2005 11:06 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Using Transaction Vision, I see that JMS always does MQGETs for every message on the queue until it gets a match, regardless of the selection criteria method.
10 messages on a queue.
Do a selective MQGET in JMS with a selector that will not match. Even with the CorrelID as the selector, I see 10 MQGETs with a browse reported by the QM.
Put message #11 on the queue, with PETER in the CorrelID. Issue the JMS get again asking for CorrelID PETER, and now I see 12 MQGETs, the last one being a destructive MQGET.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Apr 14, 2005 11:38 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Is that with server or client bindings?
The post I referenced included a comment from an IBMer that the MQ Server will always pass the header for every message to a client, and let the client figure out any filtering. Regardless of what programming environment has invoked the client, and which client it is.
If it's with server bindings... then using any selector with WebSphere MQ and JMS can lead to healthy performance problems. Or I guess I mean "unhealthy".
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sat Apr 16, 2005 5:39 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
It is a client app.
When using Correl ID as a selector, if you prefix the value with "ID:", so instead of searching for a CorrelID of PETER, you look for ID:PETER, only one MQGET is issued on the QM side.
This performs much better and is a lot less work for the QM than searching for "PETER", where a copy of every message was being returned to the client until a match was made, at which point one more destructive get needed to be issued.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Apr 16, 2005 7:16 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
The doc on JMS states that the prefix of ID is for the provider's values.
Ideally you use it when you move the message id to the correlation id and so on.
It might be counterproductive to use it when the correlation Id was a preset sorting value and not a message id. So you might be able to search for a value "PETER" but would have to search for "ID:0x....." and the hex values for "PETER"....
Does this have an impact on performance when searching by correlation ID?
The only way to know for sure is to test, and wouldn't a java ByteArray comparison be faster than a String comparison ?
 |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sat Apr 16, 2005 10:06 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Consider a queue with 100 messages with a CorrelID of various names, and message #100 has a Correl ID of PETER.
By doing a selective JMS GET with a CorrelID of PETER, Transaction Vision shows 100 MQGETs with Browse, and then one more MQGET under cursor to finally destructivly pull the message off of the queue.
If the JMS GET is for Correl ID "ID:(HexValueForPETER)", then I see only one MQGET occuring, and it is a destructive MQGET right off of bat.
101 MQGETs versus 1 MQGET. Obviously, the deeper the queue, the more important this is, and if the queue is close to zero as it should be, the less important it is.
But what I wonder is (garbled JMS terms coming from an MQ admin starting now -->), what if there is a MessageListener on the queue, waiting for message PETER? If there are more than 0 messages on the queue, the Listsner will spin top to bottom as fast as it can over and over until Message PETER aarives, issuing thousands of MQGETs with Browse. But if that listener was instead looking for Correl ID "ID:(HexValueForPETER)", I guees it would just issue a Get with Wait, and patiently wait for the message, without bombarding the QM woith MQGETs with Browse??????
I have heard the terms Message Listener and Message Receiever. How are they different, and how do they relate to the above?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Apr 16, 2005 12:48 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Let's look at the differences
a) Message Listener -- always listening pushing the message to the onMessage method
b) Message Receiver -- does a get -- with or without selector --with or without wait
The rest is all in how it is implemented....
Enjoy  |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
