| Author |
Message
|
| RB |
 Posted: Fri Jul 18, 2008 11:29 am Post subject: Issues in getting messages in Multi Threaded environment |
 |
|
Acolyte
Joined: 23 May 2006
Posts: 56
|
All,
We have a java program reading messages from the queue. We are facing issues in getting messages in some specific cases. The processing logic in the get message is given below
Public getmessage()
{
1. Open connection to the queue manager (gets the handle from an existing pool)
2. Create queue object - MQQueue mqQueue = mqQueueManager.accessQueue(“QueueName1”,openOptions,"",null,null). Options used are MQC.MQOO_INPUT_AS_Q_DEF|MQC.MQOO_BROWSE|MQC.MQOO_INQUIRE|MQC.MQOO_SET
3. Create new message object - MQMessage mqMessage=new MQMessage()
4. Display the depth of the queue using - mqQueue.getCurrentDepth();
5. Get the message - mqQueue.get(mqMessage,gmo) – Get message options used are MQC.MQGMO_SYNCPOINT| MQC.MQGMO_LOGICAL_ORDER | MQC.MQGMO_ALL_SEGMENTS_AVAILABLE|MQC.MQGMO_CONVERT
}
This method is being invoked from an external application, and the external application is multi threaded. There could be many concurrent threads running parallel, trying to get messages from different queues. Now the problem we are facing is, in some cases the get call to the queue fails without returning any messages. But the current depth shows the correct depth of the queue. Also, we know that these are committed messages and we are able to browse/read from RFHUtil. Again if we restart the application, we see that the messages are getting picked up. The issue happens couple of times a day and it is not consistent. Again, we have seen this with some specific queues (might be because of the diff in polling frequency). Though we are using MQGMO_ALL_SEGMENTS_AVAILABLE, the current scenario doesn’t have any such messages.
We are using MQ 6 and java 1.5.
Is there anything specific we need to take care of while running this in multithreaded environment? Is there any possibility of the MQ object reference pointing to the wrong queue? Any help is greatly appreciated!
Regards,
RB |
|
| Back to top |
|
 |
| sridhsri |
| Posted: Fri Jul 18, 2008 1:12 pm Post subject: |
|
|
Master
Joined: 19 Jun 2008
Posts: 297
|
Perhaps this is your case (Excerpts from contact admin):
Segments in a logical message become available for retrieval only when all segments in the logical message are available. If the queue contains segmented messages with some of the segments missing (perhaps because they have been delayed in the network and have not yet arrived), specifying MQGMO_ALL_SEGMENTS_AVAILABLE prevents retrieval of segments belonging to incomplete logical messages. However, those segments still contribute to the value of the CurrentQDepth queue attribute; this means that there might be no retrievable logical messages, even though CurrentQDepth is greater than zero. If there are no other messages that are retrievable, reason code MQRC_NO_MSG_AVAILABLE is returned after the specified wait interval (if any) has expired.
The processing of MQGMO_ALL_SEGMENTS_AVAILABLE depends on whether MQGMO_LOGICAL_ORDER is also specified:
* If both options are specified, MQGMO_ALL_SEGMENTS_AVAILABLE has an effect only when there is no current logical message. If there is a current logical message, MQGMO_ALL_SEGMENTS_AVAILABLE is ignored. This means that MQGMO_ALL_SEGMENTS_AVAILABLE can remain on when processing messages in logical order. |
|
| Back to top |
|
|
| RB |
| Posted: Fri Jul 18, 2008 1:26 pm Post subject: |
|
|
Acolyte
Joined: 23 May 2006
Posts: 56
|
Thanks for the response! But I could see from the qmgr that the messages doesnt have any segments. The flag property under segmentation has a value " Last segment MSG.mqmf_last_segment" for all the messages. Again, if it an issue with Segmentation, the program should never be able to read it. But we are noticing that a restart of the application solves the issue most of the time!
Regards,
RB |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Jul 18, 2008 2:18 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
So what is the return code and the reason code when the getter program does not pick up the messages? Are you also properly clearing the MQMD before each get?
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| RB |
| Posted: Fri Jul 18, 2008 2:38 pm Post subject: |
|
|
Acolyte
Joined: 23 May 2006
Posts: 56
|
The reason code is 2033. We are not doing anything specific to clear the MQMD, but for every get operation we are creating a new message object (and queue object as well). I am assuming it would take care of the MQMD.
The part that is confusing me is, the current depth is coming correctly. And we are not doing any match options while getting the message. So the only possible issues could be the open options not working as inteded or the MQ object pointing to the wrong queue (if by any change the multi threading is causing the current thread to refer to the queue object from some other thread!.. not sure if this is a possibility or not!)
Let me know your thoughts!
Regards,
RB |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Jul 19, 2008 12:21 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Create a new message at each loop iteration I can understand....
Opening and closing the queue is a waste of resources. Open the queue outside of the loop. Close it once the loop is finished.
Exit condition from the loop is RC=2033 (No more messages matching the criteria). Search the forum as to why using Qdepth in any indication for a get is a big nono. 
Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| RB |
| Posted: Sat Jul 19, 2008 10:16 pm Post subject: |
|
|
Acolyte
Joined: 23 May 2006
Posts: 56
|
The method is invoked by an external application and it passes the queue name as an argument. Again, the polling frequency for queues are different and controlled by the invoking application. So we had to open the queue in the method itself, though we are reusing the connection handle to the queue manager!
We are not using the depth of the queue as an indicator in this method. We were just trying to print the value, to ensure that the object is referring to the right queue. And it was displaying the count correctly. We have verified that the messages in the queue are gettable using rfhtuil, and thus it eliminates the possibilities of uncommitted messages. Still the program was not able to get a message from the queue. That's why we were suspecting the multi threads or the open options not working correctly! Any thoughts on something else that we might have missed?
Regards,
RB |
|
| Back to top |
|
|
| yalmasri |
| Posted: Sat Jul 19, 2008 11:14 pm Post subject: |
|
|
Centurion
Joined: 18 Jun 2008
Posts: 110
|
Interesting! I thought I am the only one who is facing this behavior. Have a look at this post: http://www.mqseries.net/phpBB2/viewtopic.php?t=43989
In my case, the open options are limited to MQC.MQOO_INPUT_AS_Q_DEF, which eliminates the possibility of yours being the cause.
Have you checked through MQ Explorer if any process at all is actually listening to your queue the moment your application stopped picking up messages? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Jul 20, 2008 2:48 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Also your process should not have more than 255 handles open...
Looks to me like your resource management may be off. (pairing an open with a close for the resource.)
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| zpat |
| Posted: Sun Jul 20, 2008 11:19 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| Why are you browsing the queue? Why not just get messages from it normally? Are you aware of the effect of a browse cursor? |
|
| Back to top |
|
|
| RB |
| Posted: Mon Jul 21, 2008 8:38 am Post subject: |
|
|
Acolyte
Joined: 23 May 2006
Posts: 56
|
Thanks guys for your responses!
Yalmasri, I couldn't find anything from the MQ explorer, may be because I am opening and closing the queues in the method itself.
fjb_saper, I don't think it is an issue with the resources, because even when the issue is happening for one queue, there are other threads processing messages from other queues. But next time when the issue happens, I will try to monitor the number of handles open.
zpat, though the open options has browse, we are not using it. I think I can get rid of that. We are using a get to read the message off the queue.
Regards,
RB |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Jul 21, 2008 2:27 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| RB wrote: |
zpat, though the open options has browse, we are not using it. I think I can get rid of that. We are using a get to read the message off the queue.
Regards,
RB |
Well the browse option is designed to transform your get into a ...browse!
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| yalmasri |
| Posted: Mon Jul 21, 2008 11:56 pm Post subject: |
|
|
Centurion
Joined: 18 Jun 2008
Posts: 110
|
| Quote: |
| Yalmasri, I couldn't find anything from the MQ explorer, may be because I am opening and closing the queues in the method itself. |
When you say mqQueueManager.accessQueue, then depending on the number of threads that run this method, you should see a corresponding number of processes in your explorer. When a message arrives, a process should disappear after you close the queue, and reappear after accessing it again.
When you say that your messages are being drawn down from the piling queue after a restart, that means there was something wrong with your connection that got fixed. |
|
| Back to top |
|
|
| RB |
| Posted: Tue Jul 22, 2008 5:38 pm Post subject: |
|
|
Acolyte
Joined: 23 May 2006
Posts: 56
|
Since I am opening the connection to the queue only when the external application try to poll the queue, and closes immediately after that, I practically doesn't get the time to check the status of the open counts in the explorer in between.
Today, we just tried to open the queues with the simple get options. What we have tried is, after getting 2033 error on the queue (in this case where we have messages), we opened a new connection to the queue. We used MQC.MQOO_INPUT_AS_Q_DEF alone in the open options and MQC.MQGMO_SYNCPOINT alone in the gmo (We are doing this only after getting 2033 and having depth greater than zero). And the program was able to pick the message!
I am not sure what could have caused this! Any thoughts?
Regards,
RB |
|
| Back to top |
|
|
| yalmasri |
| Posted: Tue Jul 22, 2008 10:07 pm Post subject: |
|
|
Centurion
Joined: 18 Jun 2008
Posts: 110
|
That what I was trying to tell you, and that's how I solved my problem; getting rid of the stale connection and have a new one.
In my case everything is simple and straightforward, I don't suspect any of my options to be the issue, but even though something wrong is happening to the connection. Why's that? I'm sure one of the "Jedi Council" members can figure out  |
|
| Back to top |
|
|
|
|
