| Author |
Message
|
| jkfriends |
 Posted: Thu Jan 26, 2012 9:14 am Post subject: Messages Drain too slowly |
 |
|
Novice
Joined: 09 Oct 2009
Posts: 20
|
Hi, I've searched on this forum for this but no clear solution. Here's my situation. I have a .NET (C#) program listening to a WebSPhere MQ queue. Some application on the AIX side drops messages into this queue where this listener component (C# code) drains them and process them.
Here's the code snippet that I use in C# side:
| Code: |
MQGetMessageOptions gmo = new MQGetMessageOptions();
gmo.Options = MQC.MQGMO_WAIT | MQC.MQGMO_SYNCPOINT;
gmo.WaitInterval = MQC.MQWI_UNLIMITED;
MQMessage message = new MQMessage();
queue.Get(message, gmo);
return message; |
As you can see above, the code actually listens to this queue indefinitely until it finds a message.
All works great. Now our requirements have changed and the # of messages sent to this queue will be more. So I instantiated more instances of this C# component to drain and process these messages concurrently. So I launched 10 of them. Now when 200 messages come into the queue, I thought they would be drained and processed quickly but it takes about 30 seconds to drain the whole thing. Is there anyway we can drain messages quickly or other specific options that we can set to optimize this?
Thanks in advance. |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Thu Jan 26, 2012 9:17 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
So if you have ten copies, they will all read one message at a time. So you will see the queue draining in sets of 10, more or less.
So if it takes 1.5 seconds to drain a single message, then 200 messages will take... 30 seconds. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Jan 26, 2012 9:21 am Post subject: Re: Messages Drain too slowly |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jkfriends wrote: |
| As you can see above, the code actually listens to this queue indefinitely until it finds a message. |
I hope in the actual code there's something to stop the application listening on the queue, especially if something external to the application happens (like the WMQ admin trying to close the queue manager). Because as written, that code will stop the queue manager closing neatly and you may not get Xmas cards from the admin group in future.
This has nothing to do with your problem which has been ably described by my most worthy associate.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Jan 26, 2012 9:28 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Vitor is suggesting that it is NOT a best-practice to specify MQC.MQWI_UNLIMITED. Wait unlimited will keep the qmgr from ending normally, should the sysadmin attempt to shut it down.
Instead, of wait unlimited, specify some short and reasonable time period for the the get call to end (with r/c 2033 - no message available), then reissue the get call.
So, how long does it take to process only one message?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Thu Jan 26, 2012 9:41 am Post subject: Re: Messages Drain too slowly |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| bruce2359 wrote: |
| Vitor is suggesting that it is NOT a best-practice to specify MQC.MQWI_UNLIMITED. Wait unlimited will keep the qmgr from ending normally, should the sysadmin attempt to shut it down. |
If the GMO options included MQC.MQGMO_FAIL_IF_QUIESCING, then wouldn't that allow the qmgr to end normally, even with MQC.MQWI_UNLIMITED? |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Jan 26, 2012 9:48 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| bruce2359 wrote: |
| Vitor is suggesting that it is NOT a best-practice to specify MQC.MQWI_UNLIMITED. Wait unlimited will keep the qmgr from ending normally, should the sysadmin attempt to shut it down. |
Or if you do, ensure you've included another option to match (and assist the sys admin).
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| exerk |
| Posted: Thu Jan 26, 2012 9:50 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| bruce2359 wrote: |
| Vitor is suggesting that it is NOT a best-practice to specify MQC.MQWI_UNLIMITED. Wait unlimited will keep the qmgr from ending normally, should the sysadmin attempt to shut it down. |
Or at least ensure there is a MQGMO_FAIL_IF_QUIESCING in there somewhere 
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Jan 26, 2012 9:50 am Post subject: Re: Messages Drain too slowly |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| rekarm01 wrote: |
| bruce2359 wrote: |
| Vitor is suggesting that it is NOT a best-practice to specify MQC.MQWI_UNLIMITED. Wait unlimited will keep the qmgr from ending normally, should the sysadmin attempt to shut it down. |
If the GMO options included MQC.MQGMO_FAIL_IF_QUIESCING, then wouldn't that allow the qmgr to end normally, even with MQC.MQWI_UNLIMITED? |
When you researched MQGMO_FAIL_IF_QUIESCING, what did you discover?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| jkfriends |
| Posted: Thu Jan 26, 2012 9:59 am Post subject: |
|
|
Novice
Joined: 09 Oct 2009
Posts: 20
|
| I just pasted only the core logic. I do have the logic to stop the listener gracefully if the message string is set to "STOP_SERVICE_{Guid}" where Guid is the unique generated identifier that is kept in the session when the service was first instantiated. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Jan 26, 2012 10:32 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jkfriends wrote: |
| I just pasted only the core logic. I do have the logic to stop the listener gracefully if the message string is set to "STOP_SERVICE_{Guid}" where Guid is the unique generated identifier that is kept in the session when the service was first instantiated. |
So if the sys admin wants to shutdown the queue manager it's incumbant on him to generate & send such a message to each running instance of your application?
You wouldn't get a Xmas card from me in those circumstances.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jkfriends |
| Posted: Thu Jan 26, 2012 11:31 am Post subject: |
|
|
Novice
Joined: 09 Oct 2009
Posts: 20
|
| Vitor wrote: |
| jkfriends wrote: |
| I just pasted only the core logic. I do have the logic to stop the listener gracefully if the message string is set to "STOP_SERVICE_{Guid}" where Guid is the unique generated identifier that is kept in the session when the service was first instantiated. |
So if the sys admin wants to shutdown the queue manager it's incumbant on him to generate & send such a message to each running instance of your application?
You wouldn't get a Xmas card from me in those circumstances. |
No, this is a Windows service. When you stop the service from the service manager in Windows, I drop a STOP command myself (i.e., instance) to shutdown itself. No sys admin is involved.
BTW, I've solved this issue. Instead of doing the actual processing of the message in the same transaction as fetch, I simply shifted the control to a new thread. So in essence I spawn a new thread as soon as I have a good message to process. I can't believe the speed now. All 200 messages were consumed in 2 seconds (blazing fast).
Hope this will help someone who tried to solve the same problem.
Thanks everyone. You may close this thread. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Jan 26, 2012 11:36 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jkfriends wrote: |
| No, this is a Windows service. When you stop the service from the service manager in Windows, I drop a STOP command myself (i.e., instance) to shutdown itself. No sys admin is involved. |
So there's never a situation where anyone other than you would need to close the queue manager? For any reason?
Interesting.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jkfriends |
| Posted: Thu Jan 26, 2012 11:44 am Post subject: |
|
|
Novice
Joined: 09 Oct 2009
Posts: 20
|
| Vitor wrote: |
| jkfriends wrote: |
| No, this is a Windows service. When you stop the service from the service manager in Windows, I drop a STOP command myself (i.e., instance) to shutdown itself. No sys admin is involved. |
So there's never a situation where anyone other than you would need to close the queue manager? For any reason?
Interesting. |
That's correct. The other side of the fence (AIX admin) doesn't even know that this listener can process a STOP command. It will always listen indefinitely. It there is a maintenace of MQ manager, then someone will manually stop this from the service manager - it is part of the operational procedure. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Jan 26, 2012 11:54 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jkfriends wrote: |
| The other side of the fence (AIX admin) doesn't even know that this listener can process a STOP command. It will always listen indefinitely. It there is a maintenace of MQ manager, then someone will manually stop this from the service manager - it is part of the operational procedure. |
Is this connected to a Windows queue manager or the AIX one? If the AIX, it could get interesting if an inexperienced AIX guy who doesn't know the procedure tries (and fails) to bring down the queue manager; potentially a useful April Fool for someone I know.
Anyway, I am heartened to hear you have a procedure & total confidence it will be followed.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Sat Jan 28, 2012 8:10 pm Post subject: Re: Messages Drain too slowly |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| rekarm01 wrote: |
| bruce2359 wrote: |
| Vitor is suggesting that it is NOT a best-practice to specify MQC.MQWI_UNLIMITED. Wait unlimited will keep the qmgr from ending normally, should the sysadmin attempt to shut it down. |
If the GMO options included MQC.MQGMO_FAIL_IF_QUIESCING, then wouldn't that allow the qmgr to end normally, even with MQC.MQWI_UNLIMITED? |
That was a rhetorical question.
| bruce2359 wrote: |
| When you researched MQGMO_FAIL_IF_QUIESCING, what did you discover? |
Some of the more obvious "discoveries":- Vitor may have been suggesting something else.
- MQC.MQWI_UNLIMITED need not keep the qmgr from ending normally.
- Specifying MQC.MQWI_UNLIMITED is a perfectly fine practice.
- Creating a loop for MQGET merely to avoid MQC.MQWI_UNLIMITED is a bad practice.
|
|
| Back to top |
|
|
|
|
