| Author |
Message
|
| griffel |
 Posted: Mon Oct 08, 2007 1:31 pm Post subject: Multiple access to shared queue with ConnectionConsumer |
 |
|
Novice
Joined: 22 Mar 2005
Posts: 23
|
Hi,
we have two version 6 queue managers MQ1A and MQ1B serving a clustered queue-sharing group on zOS.
We have a shared queue on them, let's say QUEUE1.
We have two Java application processes getting messages from QUEUE1 using ConnectionConsumers (IBM's ASF stuff) connected to MQ1A.
Scenario 1: We put 10000 messages to QUEUE1. Both application processes GET them which ends up with let's say application 1 got 4930 messages, application 2 got 5001 messages, and 69 message still in the queue.
Starting a third application instance GETs the remaining 69 messages (the other two instances still being active).
Scenario 2: Same as scenario 1, but now each of the both application processes is connected to a different queue manager (that's app1 connects to MQ1A and app2 connects to MQ1B). Everything works fine, all 10000 messages are removed from QUEUE1 and are processed.
Scenario 3: Same as scenario 1, but now QUEUE1 is NOT shared (only a local queue to MQ1A). Again, everything works fine, all messages are removed and processes correctly.
Question: Why does scenario 1 end up with messages remaining on the queue? Does anyone know the exact semantics and GET/BROWSE behavior of IBM ConnectionConsumer implementation?
Any help appreciated,
Frank |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Mon Oct 08, 2007 2:06 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Well Frank, you've left a lot unsaid:
| Quote: |
we have two version 6 queue managers MQ1A and MQ1B serving a clustered queue-sharing group on zOS.
We have a shared queue on them, let's say QUEUE1.
We have two Java application processes getting messages from QUEUE1 using ConnectionConsumers (IBM's ASF stuff) connected to MQ1A.
Scenario 1: We put 10000 messages to QUEUE1. Both application processes GET them which ends up with let's say application 1 got 4930 messages, application 2 got 5001 messages, and 69 message still in the queue.
Starting a third application instance GETs the remaining 69 messages (the other two instances still being active). |
You are talking about connection consumers so I guess you are using JMS.
You did not specify if you were using a messageListener or just the receiver.receive()
functionality. Nor did you specify the speed of your receiver vs the speed of your sender...
It may be quite legitimate to have 69 messages left on the queue if the queue ran empty at some time and you stopped the receiver...
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| griffel |
| Posted: Mon Oct 08, 2007 11:57 pm Post subject: |
|
|
Novice
Joined: 22 Mar 2005
Posts: 23
|
"ConnectionConsumer" is a Java Interface defined by the JMS standard (that's why I thought it might be a nice idea to post my problem to the Java/JMS discussion group ...).
A ConnectionConsumer implementation like IBM's (aka ASF) has no explicitly available receive, it just serves its "onMessage"-method using a supplied thread pool for session handling.
Therefore it is inherently concurrent anyway - that's why I'm even more surprised that adding another application process (BTW, each of them uses 5 threads with its ConnectionConsumer) result in concurrency problems.
All 10000 messages are put to the queue. The applications are started after this and stay up. BTW, if I put another message to the queue when there are some messages remaining they are all caught and process. Thus, the application proccesses seem to work normally. It looks like the consumers "get out of synch" / "forget to browse again" or the like.
BTW, no selectors are used - just get everything from the queue. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Oct 09, 2007 1:51 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Now we are getting somewhere.
It looks to me that you might be writing your own session pools/connection pools or not using the jndi setup according to best practice.
There is an old but excellent document about JMS topology (IBM redbooks) that will give you a much better insight about the concurrency issues.
Basically each MDB assigned to a queue should have it's own connection. Each instance of the MDB on that same queue should have it's own connection. The why is about multithreading and performance...
Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| griffel |
| Posted: Wed Oct 10, 2007 12:27 am Post subject: |
|
|
Novice
Joined: 22 Mar 2005
Posts: 23
|
Sorry, but ...
JNDI is not about session handling. Actually, you're right - we don't
use JNDI (just adds another complexity and its not needed if you just want to get a factory for your connections).
IBM's implementation of ConnectionConsumer needs a "self made" session pool handling. We use the example implementation delivered by IBM with some minor enhancements.
We do not use MDBs (just another useless complexity ...) - just plain Java and the JMS/MQ packages. And: Our systems run very well for the last three years - its just that we now want to go shared-queues to get real availability. We're quite experienced with all that stuff, in fact invented some of it ourselves. Maybe, our mistake was to use a piece of pre-fabricated code (IBMs ASF) instead of writing our own receiver/consumer. Now we need in-depth knowledge about this specific implementation.
Anyway, thanks for contemplating,
Frank |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Oct 10, 2007 2:15 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Ok.. So the big problems you're going to run into is when you either stop/close the connection and the session/session consumer is still open and has registered MessageListeners....
Not a nice picture...
If you need more pointers let me know...
On top of this if you want true concurrency you should be using mutliple connections. The connection will synchronize the put/get to/from the same destination...
Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| griffel |
| Posted: Wed Oct 17, 2007 7:32 am Post subject: SOLVED |
|
|
Novice
Joined: 22 Mar 2005
Posts: 23
|
Just in case someone reads this in search of solutions to similar problems:
Setting the property com.ibm.mq.jms.tuning.QATVersion=1 on the Java client side cause the MQQueueAgent to use the old style mechanism and everything works fine. Thus, its a bug fixed by IBM's latest patches for MQ V6/zOS (needs at least level 6.0.2.1 on the Java side). |
|
| Back to top |
|
|
|
|
