| Author |
Message
|
| PeterPotkay |
 Posted: Thu Dec 31, 2009 5:00 am Post subject: |
 |
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
zpat, circular or linear in your case?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
 |
| Michael Dag |
| Posted: Thu Dec 31, 2009 5:01 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
I would not delete the log files yourself, MQ should "see" the extra 10 logs as obsolete or no longer needed and clean them up.
as long as there are references to anything in those 10 extra logs that are needed... MQ won't like it when you deleted them...
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Dec 31, 2009 5:04 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
When does it actually remove them? - as we need the disk space back!
Can we force it to perform this housekeeping? |
|
| Back to top |
|
|
| mvic |
| Posted: Thu Dec 31, 2009 5:08 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| zpat wrote: |
| Delete the last 10 log files (QM originally created with 30). |
Circular or linear?
If circular, this bit is definitely not right. Mostly because you can't reliably tell what log files in the ring are needed for restart. Leave MQ to delete unused files.
If linear, continue to observe the messages in the qm error logs saying which ones can/can't be removed. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Dec 31, 2009 5:09 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| Sorry, I should have said - circular logs. |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Thu Dec 31, 2009 5:20 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
| zpat wrote: |
| Sorry, I should have said - circular logs. |
then you are stuck to having MQ get rid of the logs...
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Dec 31, 2009 11:50 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
zpat, I just did the same thing for a Circular QM and the QM was not reducing the logs. I opened a PMR, round and round we went. Eventually we figured it out.
Assume 1 MB log files. You have 30 and drop the number to 20.
Once you put > 30 MB of persistent data thru the queue, the QM will drop log #29.
Once you put > 29 MB of persistent data thru the queue, the QM will drop log #28.
Once you put > 28 MB of persistent data thru the queue, the QM will drop log #27.
In my case I had the log file size as big as they could be, lots of those logs and very little spare disk space to allow queueing of all that persistent data. After deleting everything I possible could on the server I was just able to get enough persistent data queued to delete one log. I cleared the queue and repeated the process over and over. Towards the end I was able to queue enough data each time to clear up multiple logs.
If you are starting with a lot of space where your queue files live you may be able to queue enough persistent data in one shot to loop thru the logs enough times to clean them up in one shot.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mvic |
| Posted: Thu Dec 31, 2009 11:59 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| PeterPotkay wrote: |
| If you are starting with a lot of space where your queue files live you may be able to queue enough persistent data in one shot to loop thru the logs enough times to clean them up in one shot. |
Or, if not, there is an equivalent alternative: put one 10 Mb persistent message onto a queue, then get it. Then repeat. Result: consumption of log space, roughly 10 Mb for each put and a little bit (100s of bytes?) for each get. Also no net growth in the q space needed. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Dec 31, 2009 12:55 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| mvic wrote: |
| PeterPotkay wrote: |
| If you are starting with a lot of space where your queue files live you may be able to queue enough persistent data in one shot to loop thru the logs enough times to clean them up in one shot. |
Or, if not, there is an equivalent alternative: put one 10 Mb persistent message onto a queue, then get it. Then repeat. Result: consumption of log space, roughly 10 Mb for each put and a little bit (100s of bytes?) for each get. Also no net growth in the q space needed. |
I did try that and it didn't work. Although now that I think of it I may have been getting as I was putting and the persistent messages were bypassing the logs due to a waiting getter. I would guess that as long as your put ends and THEN your MQGET starts and ends it will work if repeated enough times to cycle thru the logs.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| zpat |
| Posted: Fri Jan 01, 2010 10:18 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Thanks, sounds like hard work, but good to know the answer...
My logs are pretty large at 16384 pages each - so this could be a challenge! |
|
| Back to top |
|
|
| zpat |
| Posted: Mon Jan 04, 2010 1:20 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| PeterPotkay wrote: |
| I did try that and it didn't work. Although now that I think of it I may have been getting as I was putting and the persistent messages were bypassing the logs due to a waiting getter. |
I thought that persistent messages always went to the log - even with a waiting getter? |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Jan 04, 2010 5:35 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
I can't find the reference in the InfoCenter or in a Technote article that deals with this topic, although I know I've seen it, to verify whether it applies to both persistent and non persistent messages put to outstanding gets outside of syncpoint.
Although I put thousands upon thousands of huge messages from my putting application to the waiting getting application and didn't get the results I expected, which lead me to that conclusion.
I wish I could find the reference to this topic, its driving me nuts not being able to find it.
Think about it though. If the message is being MQPUT to a queue where an MQGET is already waiting that will consume that message, and the put and get are both outside of syncpoint, of what benefit is there to write the message to the log, whether its persistent or not? In my mind I think if they want MQ to perform as well as it can, that in this case they would skip the write to logs and the disk I/O penalty. As soon as you add syncpoint then the equation totally changes and the writes to the logs become mandatory, because MQ can't know when the MQBACK or MQCMIT will occur.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mvic |
| Posted: Mon Jan 04, 2010 5:59 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| PeterPotkay wrote: |
| Think about it though. If the message is being MQPUT to a queue where an MQGET is already waiting that will consume that message, and the put and get are both outside of syncpoint... |
I know I'm very off topic now, but it's a bad application pattern to deal with persistent messages outside syncpoint.
Perhaps everyone reading this knows this fact. But if not, then it was worth me saying it.
Not only reliability, but performance will improve if you do persistent messaging inside syncpoints. |
|
| Back to top |
|
|
| golam1983 |
| Posted: Fri Feb 05, 2010 12:46 pm Post subject: |
|
|
Apprentice
Joined: 27 Jun 2008
Posts: 35
|
I usually handle this type of situation, keeping a huge no of secondary log files like primary 20 and secondary 100  |
|
| Back to top |
|
|
|
|
