| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| MQ + WAS + JMS and queue handle pooling |
« View previous topic :: View next topic » |
| Author |
Message
|
| hopsala |
 Posted: Sat Jan 08, 2011 1:07 pm Post subject: MQ + WAS + JMS and queue handle pooling |
 |
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
So, we all know that the most costly operations in MQ are MQCONN and MQOPEN. MQCONN due to the handshaking protocol (+SSL if valid) and MQOPEN mainly due to OAM security checks. To reduce latency, WAS has both a connection factory pool and a general mq connection pool at its disposal, from which you get an already-open connection, if there's one available. But what about queue handles? Is there no pooling for queue handles?
Assume the following code (Adapted from the Hursley MQ blog):
| Code: |
MQQueueConnection connection = (MQQueueConnection) cf.createQueueConnection();
MQQueueSession session = (MQQueueSession) connection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);
MQQueue queue = (MQQueue) session.createQueue("queue://QM1/Q1");
MQQueueSender sender = (MQQueueSender) session.createSender(queue);
MQQueueReceiver receiver = (MQQueueReceiver) session.createReceiver(queue);
JMSTextMessage message = (JMSTextMessage) session.createTextMessage("SimplePTP "+ uniqueNumber);
connection.start();
sender.send(message);
sender.close();
session.close();
connection.close() |
Let's say APP1 runs this code. Behind the scenes WAS MQCONNs to QM1, MQOPENs Q1 and MQPUTs a message. APP1 then ends gracefully closing the session and connection, thereby returning the connection to QM1 to the pool. Then APP2 (or maybe APP1 again) runs the exact same code, and let's assume it gets the same QM1 handle - Will it also get the same Q1 handle or will another be created, alongside OAM security checks again?
I'd love to get an answer for this. I've been looking everywhere to no avail. I could, of course, start experimenting with WAS and MQ and keep watch on Open Handle Count, but I'd rather get the official version before I start reverse-engineering.
Also, if someone could explain to me exactly when MQOPEN is performed in JMS I'd much appreciate it. Is it that in connection.start() all queue handles referred are open? Is it in sender.send(msg)?
Much obliged!  |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Sat Jan 08, 2011 11:48 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Queue handles are allocated once you have declared them to the session in JMS. So your code Queue myqueue = Session.createQueue("queue://QMGR/QNAME"); is the allocation of the queue handle. The usage you will do with the queue handle (i.e. open options) get determined later down the road when you use the queue with a sender or receiver.
You have also the anonymous sender which will use a put1 and for which you need to pass the destination to the send method.
Whichever way you use a destination, the scope of the destination for an open call is the session (transaction scope). However once defined by a session the destination object can be used across sessions and the scope of the object itself (caching) is that of the regular scope of such an object. To be usable (put, get, browse) you have to be in a session scope.
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| hopsala |
| Posted: Mon Jan 10, 2011 10:46 pm Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| fjb_saper wrote: |
| Queue handles are allocated once you have declared them to the session in JMS. So your code Queue myqueue = Session.createQueue("queue://QMGR/QNAME"); is the allocation of the queue handle. The usage you will do with the queue handle (i.e. open options) get determined later down the road when you use the queue with a sender or receiver. |
Thanks for the reply fjb. Now, I don't understand exactly how what you're describing is possible. If I remember my MQI correctly, you have to specify open options before issuing MQOPEN, not after. This is due to security checks which take place upon opening the queue. Only after MQOPEN is issued are you given a queue handle. (I'm assuming it's still MQI under the covers in JMS, right?)
Also, how can queue handle be allocated on session.createQueue(..) if I haven't issued connection.start() yet? I mean, I could set the connection parms only after the createQueue() call and still the code would work..
| fjb_saper wrote: |
| Whichever way you use a destination, the scope of the destination for an open call is the session (transaction scope). However once defined by a session the destination object can be used across sessions and the scope of the object itself (caching) is that of the regular scope of such an object. To be usable (put, get, browse) you have to be in a session scope. |
Well, I'm not really using a Destination object, since this is JMS 1.0 code and not JMS 1.1 . Is the scope of queue handles different in either version?
In any event, what you're saying, if I understand you correctly, is that I'll have to give my Destination object manually to other servlets to reuse the same queue handle. I was asking if WAS was managing queue handles in a similar way to connection handles, so that this is done automatically and transparently. I think you're implying that the answer is no.. yes? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Jan 11, 2011 1:33 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| hopsala wrote: |
| fjb_saper wrote: |
| Queue handles are allocated once you have declared them to the session in JMS. So your code Queue myqueue = Session.createQueue("queue://QMGR/QNAME"); is the allocation of the queue handle. The usage you will do with the queue handle (i.e. open options) get determined later down the road when you use the queue with a sender or receiver. |
Thanks for the reply fjb. Now, I don't understand exactly how what you're describing is possible. If I remember my MQI correctly, you have to specify open options before issuing MQOPEN, not after. This is due to security checks which take place upon opening the queue. Only after MQOPEN is issued are you given a queue handle. (I'm assuming it's still MQI under the covers in JMS, right?)
Also, how can queue handle be allocated on session.createQueue(..) if I haven't issued connection.start() yet? I mean, I could set the connection parms only after the createQueue() call and still the code would work..
|
I would like to see how you set the connection after the createQueue() call as the createQueue call is a session scope call. You will need to have a valid session to create a queue (exception is retrieving the queue from JNDI). You can however reuse the created Queue (i.e. Destination ) in a different connection of the same provider.
| hopsala wrote: |
| fjb_saper wrote: |
| Whichever way you use a destination, the scope of the destination for an open call is the session (transaction scope). However once defined by a session the destination object can be used across sessions and the scope of the object itself (caching) is that of the regular scope of such an object. To be usable (put, get, browse) you have to be in a session scope. |
Well, I'm not really using a Destination object, since this is JMS 1.0 code and not JMS 1.1 . Is the scope of queue handles different in either version?
In any event, what you're saying, if I understand you correctly, is that I'll have to give my Destination object manually to other servlets to reuse the same queue handle. I was asking if WAS was managing queue handles in a similar way to connection handles, so that this is done automatically and transparently. I think you're implying that the answer is no.. yes? |
This has nothing to do with JMS 1.0 / JMS 1.1. A javax.jms.Queue is per definition an instance of javax.jms.Destination. You might try and think a little less about analogies with java base and more about how the JMS model works.
I am not implying anything there. All I am saying is that you do have a reusable object in the Destination. What happens behind the scenes, and if that object is only being used to create/recreate the handle is done in the proprietary code of the IBM JMS implementation.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Jan 11, 2011 1:38 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
As you posted in the Performance / Monitoring forum it might be of advantage to let us know where you are trying to go / lead with this.
Typically the monitoring on your WAS would be method based and pool based (JMS connection and pool usage) and the monitoring on the MQ side does not change whether you use JMS or not.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| hopsala |
| Posted: Thu Jan 13, 2011 1:38 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| fjb_saper wrote: |
| hopsala wrote: |
| Also, how can queue handle be allocated on session.createQueue(..) if I haven't issued connection.start() yet? I mean, I could set the connection parms only after the createQueue() call and still the code would work.. |
I would like to see how you set the connection after the createQueue() call as the createQueue call is a session scope call. You will need to have a valid session to create a queue (exception is retrieving the queue from JNDI). You can however reuse the created Queue (i.e. Destination ) in a different connection of the same provider. |
Yeah, you're right. Had a brain surge there for a moment. But I still don't understand how you can determine whether the queue is open for INPUT or OUTPUT after the MQOPEN is issued:
| hopsala wrote: |
| fjb_saper wrote: |
| Queue handles are allocated once you have declared them to the session in JMS. So your code Queue myqueue = Session.createQueue("queue://QMGR/QNAME"); is the allocation of the queue handle. The usage you will do with the queue handle (i.e. open options) get determined later down the road when you use the queue with a sender or receiver. |
If I remember my MQI correctly, you have to specify open options before issuing MQOPEN, not after. This is due to security checks which take place upon opening the queue. Only after MQOPEN is issued are you given a queue handle. (I'm assuming it's still MQI under the covers in JMS, right?) |
?
| fjb_saper wrote: |
| hopsala wrote: |
| Well, I'm not really using a Destination object, since this is JMS 1.0 code and not JMS 1.1 . Is the scope of queue handles different in either version? |
This has nothing to do with JMS 1.0 / JMS 1.1. A javax.jms.Queue is per definition an instance of javax.jms.Destination. You might try and think a little less about analogies with java base and more about how the JMS model works. |
Sorry, than I misunderstood what you meant. You meant Destination class as extended by Queue, and you're right. What threw me off is that in JMS 1.1 you use Destination explicitly in your code, whereas in 1.0 you'd use either Queue or Topic (http://publib.boulder.ibm.com/infocenter/wmqv7/v7r0/topic/com.ibm.mq.csqzaw.doc)
| fjb_saper wrote: |
| I am not implying anything there. All I am saying is that you do have a reusable object in the Destination. What happens behind the scenes, and if that object is only being used to create/recreate the handle is done in the proprietary code of the IBM JMS implementation. |
Yeah, but that's what I was asking, wasn't it? and this is no more "proprietary knowledge" than connection pooling configuration. If there's a queue handles pool behind the scenes, it's something the user needs to know about. My question is a feature query - does WAS support queue handle pooling? |
|
| Back to top |
|
|
| hopsala |
| Posted: Thu Jan 13, 2011 1:51 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| fjb_saper wrote: |
As you posted in the Performance / Monitoring forum it might be of advantage to let us know where you are trying to go / lead with this.
Typically the monitoring on your WAS would be method based and pool based (JMS connection and pool usage) and the monitoring on the MQ side does not change whether you use JMS or not. |
I was going for the "Performance" bit rather than the "Monitoring" one.
Here's the scenario:
I have a JMS wrapper class some guy wrote, who was far from being MQ-savvy. The send() method looks like the code I posted above - that is, new Connection, Session, Queue and Sender classes are created on each send() and closed at the end of the method. Not what you'd call a high-throughput sort-of design..
Changing it is a bit of a hassle, for many reasons you can imagine. Considering that WAS pools my connections for me, creating a new Connection class each send() is not that bad, because there's no MQCONN issued, I'm just getting it from the pool (although obviously it's not as fast as only closing the connection when I'm done sending). So that settles MQCONN for me; No rewriting required.
As for MQOPEN, if there's an MQOPEN call issued at every send() then I still have a major performance problem on my hands. If, on the other hand, WAS pools my queue handles for me, than I can keep the code as it is, and only pay the minor fee of constantly popping queue and connection handles in and out of the two pools.
So I brought this up to check whether I should to rewrite the wrapper class or not. If I know there's no queue handle pooling, I'll either pool the handles myself (unlikely) or require that the programmer issue three separate calls - open, send and close. (which is what I'd make him do if I had written the wrapper to begin with) |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Jan 13, 2011 2:51 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
I would take the tact that it should be rewritten "because it doesn't follow JMS best practices" and make no mention of MQ.
I would also answer the questions you have by running an MQ client trace... |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Jan 13, 2011 1:10 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
As regards to your open options on the queue.
My guess is that if you have a repetitive operation you need to write your MDB in such a way that you cache the connection, session as well as the sender or receiver and access it through a getter method.
This would avoid creating a new connection and session and receiver or sender every time around. The getter method allows you to create the artifacts the first time around and then reuse them.
Of course you need to look into the lifecycle methods and maybe close and nullify the respective fields as a resource clean up on passivation...
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| hopsala |
| Posted: Sun Jan 16, 2011 2:11 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| mqjeff wrote: |
| I would take the tact that it should be rewritten "because it doesn't follow JMS best practices" and make no mention of MQ. |
In general I agree. My only concern is that the purpose of the JMS wrapper is to simplify working with JMS (personally I find it relatively straightforward, but most programmers find it difficult to grasp), and if I don't pool any resources then wrapping is useless. I'll probably find something in-between - pool queue handles in a local object (not a singleton), and restrict the user to one connection and one session. If someone wants the full range of features, transactionality etc, he'll just have to go pure JMS.
| mqjeff wrote: |
| I would also answer the questions you have by running an MQ client trace... |
I guess I'll just have to. It's a shame IBM didn't publish a simple table corresponding JMS and MQI calls. I'll send a literature feedback.
| fjb_saper wrote: |
| Of course you need to look into the lifecycle methods and maybe close and nullify the respective fields as a resource clean up on passivation... |
Problematic, since java AFAIK doesn't have a destructor method equivalent. There's only finalize(), which is not always called (http://www.codeguru.com/java/tij/tij0051.shtml). I'll just have to hope that the user doesn't forget to issue close()..
Thanks! |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Jan 16, 2011 12:58 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| hopsala wrote: |
| fjb_saper wrote: |
| Of course you need to look into the lifecycle methods and maybe close and nullify the respective fields as a resource clean up on passivation... |
Problematic, since java AFAIK doesn't have a destructor method equivalent. There's only finalize(), which is not always called (http://www.codeguru.com/java/tij/tij0051.shtml). I'll just have to hope that the user doesn't forget to issue close()..
Thanks! |
The whole point here was to do the resource cleanup (i.e. issue explicit close and set fields to null) when the MDB's passivation method was being called...
On activation you need nothing to do. On first use after activation you would automatically recreate through the getter methods the artifacts needed.
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| hopsala |
| Posted: Tue Jan 18, 2011 2:59 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| fjb_saper wrote: |
The whole point here was to do the resource cleanup (i.e. issue explicit close and set fields to null) when the MDB's passivation method was being called...
On activation you need nothing to do. On first use after activation you would automatically recreate through the getter methods the artifacts needed.
Have fun |
True, except we're not working with an MDB listener, and the wrapper is supposed to work standalone sans-WAS as well. We have been thinking of going that route lately, due to this reason and many others, but for now it's just too much work. As usual, project deadlines beat correct design principles... |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
