| Author |
Message
|
| BBM |
 Posted: Mon Jan 30, 2006 4:31 am Post subject: Guidelines for Maximum Queue Depth |
 |
|
Master
Joined: 10 Nov 2005
Posts: 217
Location: London, UK
|
Hi,
I've recently inherited two 5.3.7 queue managers running on W2K boxes.
I'm aware that some of the max queue depths on the boxes are different to others e.g. some are 500 some are 400000.
I'm aware of the rationale behind setting max queue depths to different values.
My question is, is there any detrimental effect by having a very large max queue depth?
I'm aware that MQ for Windows has some resource limitations (desktop heap) etc, but this should not affect a high queue depth as they are all persistent messages and should be stored in the logs.
I'm striving to get proper monitoring in place so that we catch queue build up early - but until then I'm not keen on fishing messages out of the DLQ (the DLQ handler doesn't work on our QM's due to non-standard headers - don't ask).
Many thanks as always for any help. |
|
| Back to top |
|
 |
| csmith28 |
| Posted: Mon Jan 30, 2006 10:13 am Post subject: |
|
|
Grand Master
Joined: 15 Jul 2003
Posts: 1196
Location: Arizona
|
Much of what you are asking can only be determined by doing a detailed evaluation of the amount of traffic, message size and the amount of space available in your ~\QMGR\active directory.
There is no one answer. If you are dealing with 500 10meg messages the anwer is different than if you are dealing with 5000 1meg messages.
_________________
Yes, I am an agent of Satan but my duties are largely ceremonial. |
|
| Back to top |
|
|
| JT |
| Posted: Mon Jan 30, 2006 12:28 pm Post subject: |
|
|
Padawan
Joined: 27 Mar 2003
Posts: 1564
Location: Hartford, CT.
|
| Quote: |
| My question is, is there any detrimental effect by having a very large max queue depth? |
A "run-away" application comes to mind. Without a legitimate restrictive limit, you're risking the systems available resources at the peril of other well-behaving applications |
|
| Back to top |
|
|
| markt |
| Posted: Mon Jan 30, 2006 1:03 pm Post subject: |
|
|
Knight
Joined: 14 May 2002
Posts: 508
|
| Simply defining a high maxdepth does not cause any extra resources to be used - so nothing detrimental there. It's only when the messages start to hit the queue that it might hurt. |
|
| Back to top |
|
|
| csmith28 |
| Posted: Mon Jan 30, 2006 1:11 pm Post subject: |
|
|
Grand Master
Joined: 15 Jul 2003
Posts: 1196
Location: Arizona
|
| markt wrote: |
| Simply defining a high maxdepth does not cause any extra resources to be used - so nothing detrimental there. It's only when the messages start to hit the queue that it might hurt. |
Indeed, a huge volume of messages sitting on the MQManager can not only impact the active logs but fill up the /var/mqm/qmgrs filesystem.
_________________
Yes, I am an agent of Satan but my duties are largely ceremonial. |
|
| Back to top |
|
|
| mvic |
| Posted: Mon Jan 30, 2006 1:38 pm Post subject: Re: Guidelines for Maximum Queue Depth |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| BBM wrote: |
| My question is, is there any detrimental effect by having a very large max queue depth? |
Analyse your business rules, your message sizes, and your disk space.
If there is a business need to store (say) 1 million messages in case your drain processor is going slow, then fine.
But if each of those messages is a megabyte in length then there will be trouble because you will have about 1000 Gb of data, which can cause problems for backups, rcdmqimg and so on.
If you feel comfortable doing so, could you share some data about the requirements you are trying to fulfill? I mean, what size are the messages, what Q depth do your business rules demand? What disk space constraints are you working to? What backup / rcdmqimg policy do you have? |
|
| Back to top |
|
|
| mvic |
| Posted: Mon Jan 30, 2006 1:46 pm Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| csmith28 wrote: |
| If you are dealing with 500 10meg messages the anwer is different than if you are dealing with 5000 1meg messages. |
What difference might one see between the 2 cases above?
Both of these cases take a Q file to about 5 Gb. As long as the disk is quick, this could be OK. However, I've seen problems at higher sizes (30+ Gb) when trying to run rcdmqimg. |
|
| Back to top |
|
|
| csmith28 |
| Posted: Mon Jan 30, 2006 2:14 pm Post subject: |
|
|
Grand Master
Joined: 15 Jul 2003
Posts: 1196
Location: Arizona
|
| mvic wrote: |
| csmith28 wrote: |
| If you are dealing with 500 10meg messages the anwer is different than if you are dealing with 5000 1meg messages. |
What difference might one see between the 2 cases above?
Both of these cases take a Q file to about 5 Gb. As long as the disk is quick, this could be OK. However, I've seen problems at higher sizes (30+ Gb) when trying to run rcdmqimg. |
Depends on the value of MAXDEPTH for the Queue in question. Wasn't that the topic.
_________________
Yes, I am an agent of Satan but my duties are largely ceremonial. |
|
| Back to top |
|
|
| BBM |
| Posted: Tue Jan 31, 2006 2:23 am Post subject: |
|
|
Master
Joined: 10 Nov 2005
Posts: 217
Location: London, UK
|
Hi,
Thanks for all the replies. much appreciated.
Our normal message size is 2-4k and the reason I posed the question is that on Friday night we saw a critical queue's depth increase to 70,000 messages. The MAXDEPTH was set to 100,000 so we were quite close to the mark.
I've been assured that this activity was a "one-off" - but I am interested in increasing the MAXDEPTH to give me a safety buffer in a similiar circumstance.
I think the main point I've taken is that I need to go back to the business and stress that MQ is not a database and shouldn't be treated as such. If we do not have sufficient drain processes to process the current level of messages then we will have to upgrade.
I also didn't consider the impact to the Websphere MQ\qmgrs file system which I will now go away and evaluate.
Many thanks for your help.. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Jan 31, 2006 6:23 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
The only way to be truly safe is to determine how much disk space you have available for MQ, subtract space dedicated for MQ logs, then make sure the total of (Max Depth x Max Message Size) x all your queues is under tha number. If you do the math, you will quickly find that those numbers are to restrictive.
So you have to assume that all the queues won't fill up all at the same time. Typically, I insure that any ONE queue's Max Depth x Max Message Lenght won't fill up my available disk. Monitor EVERY queue. Hope that multiple apps don't freak out at the same time and fill their queues concurrently.
Constant arguement with the bosses "You want how much storage?!?! Look, you are barely using what you have now on this dinky old machine!" With MQ, you plan for when things are going wrong (queues backing up), not for when everything is zooming thru.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
|
|
