| Author |
Message
|
| Gideon |
 Posted: Thu Apr 05, 2012 10:35 am Post subject: Large queue sizes |
 |
|
Chevalier
Joined: 18 Aug 2009
Posts: 403
|
I have a large non-persistent queue that spilled into disk
| Code: |
find /var -size +200000c -ls| awk ' { print $7,$NF } ' | sort -rn | more
515163136 /var/mqm/qmgrs/BBB/queues/INPUT/q |
Is it possible to shrink the physical size of the queue after the messages have been drained by an application
If so how would I do it |
|
| Back to top |
|
 |
| PeterPotkay |
| Posted: Thu Apr 05, 2012 10:45 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Add one dummy message to the queue, then issue the clear q command.
Or restart the QM.
Or wait and eventually the QM will shrink the size when its good and ready.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Apr 05, 2012 10:48 am Post subject: Re: Large queue sizes |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Gideon wrote: |
| Is it possible to shrink the physical size of the queue after the messages have been drained by an application |
Yes.
| Gideon wrote: |
| If so how would I do it |
Grab a cup of coffee & wait. After a non-configurable interval, the queue manager will reclaim the space.
If this queue is filling so much disk it's causing a problem, you need to buy more disk.
Seriously. If you're that close to the edge a slightly larger number of non-persistent messages spilling out could run the queue out of space & cause a problem. Likewise, if there was some kind of queue compressing utility how would you determine when to run it? Squeezing the file at the wrong time could seriously degrade performance, which is why currently the queue manager picks its moment.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Apr 05, 2012 10:50 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| PeterPotkay wrote: |
| Add one dummy message to the queue, then issue the clear q command. |
Would this actually shrink the file size? I always thought that it just emptied the queue.
Things you learn. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Apr 05, 2012 10:52 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| PeterPotkay wrote: |
| Or restart the QM. |
I didn't mention it as typically uptime is more important than file size. But this would do it.
Or possibly I'm just to idle to go to the trouble of a stop/start when all I need to do is sit back... 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Gideon |
| Posted: Thu Apr 05, 2012 12:44 pm Post subject: |
|
|
Chevalier
Joined: 18 Aug 2009
Posts: 403
|
The restart is a good iea
Would that shrink the queue file size of an empty persistent queue as well (restarting the qmgr) ? |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Apr 06, 2012 4:14 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Gideon wrote: |
| Would that shrink the queue file size of an empty persistent queue as well (restarting the qmgr) ? |
For the upteenth time on this forum:
There is not such thing as a persistent queue!
There is a queue where the default persistence is persistent, but persistence is a message attribute. On any queue except a temporary dynamic one you can freely mix persistent & non-persistent messages.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Apr 06, 2012 5:58 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Yeah, that Clear Q command will shrink the file size, but only if there is at least one message for it to clear. Issuing it against an empty queue doesn't force the file shrinkage. At least it didn't when I last tested this on MQ 6.
Th eproblem with waiting for the QM to do it on its own time is sometimes you don't have that luxury. For whatever reason the q loaded up with a ton of messages pushed you dangerously close to a disk full condition. The queue has now been drained by the app. Yet you are stuck with an almost full file system. You can't restart the QM. You can't clear the queue. You can't wait. Sure wish we had an admin command to tell the QM to go clean up.
I stocked my QM dirs with 4 or 5 dummy files, each about 500 MB is size, in a folder called DummyFillerFiles, with a read me that explained one or more of these files could be safely deleted in a space saving emergency, just please put them back once the real space hog (Piiiigs...Innnn....Spaaaace!) was identified and cleaned up.
So here's a question I've always wanted to debate. When you size all your queues's Max Message Size x Max Queue Depth, do you size them all assuming one day you will have the worst case scenario where all queues fill up concurrently? That odds of that happening are slim, and it would seem you would have to plan for a ton of space that you would likely never come even close to using. Well, I gues depending on the app topology it might be more than a slim change. One QM, one hundred queues, 100 sending apps, one receiving app. If that one receiving app dies, its possible all the queues could eventually fill up.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Apr 06, 2012 6:53 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| PeterPotkay wrote: |
| Sure wish we had an admin command to tell the QM to go clean up. |
Yes, but it would be as abused as the REFRESH CLUSTER command.
| PeterPotkay wrote: |
| (Piiiigs...Innnn....Spaaaace!) |
| PeterPotkay wrote: |
| So here's a question I've always wanted to debate. When you size all your queues's Max Message Size x Max Queue Depth, do you size them all assuming one day you will have the worst case scenario where all queues fill up concurrently? |
Yes, balanced with the business impact of a given queue filling. Also max out xmitqs & DLQs. Disc tends to be the cheapest of the sizing resources.
Another aspect is how the site views this. It's within my experience that the queues are sized on "how long it takes us to notice a problem and shut down the sending systems".
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Apr 06, 2012 7:43 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Or, this can be viewed as a business process rule: if you are processing payroll time cards, one per employee each payroll period, then max depth should allow for each employee to submit one time card. Reality sets in here, as sometimes it takes two or three tries to get it right.
So, for example: 1000 employees x 3 tries each = maxdepth 3000 for the time card queue.
Max msg length, like max depth, comes from developers. In the case of message length, take what the developers tell you (100 bytes each message, then multiply by some fudge-factor - like 100%. Set maxmsglength to 200. This will allow for message length expansion over time.
There is no logical or practical reason to set max depth to 999,999,999; or to set max msg length to 100MB. This would just entice app developers to create programs that include infinite loops that fill queues and filesystems.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
|
|
