| Author |
Message
|
| Volvic |
 Posted: Fri Jan 15, 2010 7:50 am Post subject: LL and long running transactions? |
 |
|
Apprentice
Joined: 14 Oct 2009
Posts: 30
|
Hi there,
on windows I found that we configured 20 primary and 10 secondary log files and 256 log file pages. We use linear logging.
Are the options for primary and secondary logging even relevant for LL as the log file amount grows endlessly without manual intervention? We currently have over 3000 log files in the 'active' directory, and not 10 + 20 = 30 like we should have with CL (circular logging).
With the above configuration, are long running transactions in danger? Or is it irrelevant for LL how many primary and secondary log files are in place?
_________________
Volvic |
|
| Back to top |
|
 |
| Vitor |
| Posted: Fri Jan 15, 2010 7:59 am Post subject: Re: LL and long running transactions? |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Volvic wrote: |
| Are the options for primary and secondary logging even relevant for LL as the log file amount grows endlessly without manual intervention? |
Yes. Once all the primary & secondary is exhausted the queue manager will behave exactly like it does when a circular log is full, i.e. start cancelling transactions to free up space.
| Volvic wrote: |
| We currently have over 3000 log files in the 'active' directory, and not 10 + 20 = 30 like we should have with CL (circular logging). |
Sounds like you need to do some maintenance. The number of logs may grow endlessly without manual intervention but unless your server can add disc space in the same way sooner or later you're going to run out of space!
Also having large numbers of log files impacts queue manager start up time.
| Volvic wrote: |
| With the above configuration, are long running transactions in danger? |
Yes.
| Volvic wrote: |
| Or is it irrelevant for LL how many primary and secondary log files are in place? |
No.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mvic |
| Posted: Fri Jan 15, 2010 8:37 am Post subject: Re: LL and long running transactions? |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| Vitor wrote: |
| Also having large numbers of log files impacts queue manager start up time. |
One might expect that to be the case, but it ain't so. Checkpoint processing moves the "active" portion of the log forwards, partly to make restart time as quick as possible, but also to recover log space which is necessary too.
So if you end the queue manager normally (this performs a checkpoint) there should be almost nothing to replay when restarting. If, on the other hand, your queue manager ends badly (crash, etc.) then if checkpoint ran recently before that point, the replay will be quick. If it didn't run recently, it may be less quick.
So the important thing for restart time is not the number of log files, but the number that have been used since the last checkpoint.
If there is a long running transaction, the checkpoint can't move the active portion forwards. This situation can't carry on forever of course, and as you said, when the log is becoming full, the oldest transactions can be rolled back. Then the checkpoint processing can reclaim the space used by that transaction and things can move forward again. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jan 15, 2010 8:45 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Even if no media image has been taken and the queue manager's recovering all the way from the 1st file to the 3000th?
I stand by my comment about disc space though.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mvic |
| Posted: Fri Jan 15, 2010 8:47 am Post subject: Re: LL and long running transactions? |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| Volvic wrote: |
| With the above configuration, are long running transactions in danger? |
How long should transactions be uncommitted on your queue manager? How many bytes of persistent messages are expected to be put/got within that time?
| Quote: |
| Or is it irrelevant for LL how many primary and secondary log files are in place? |
The number of primary + secondary log files multiplied by the size of each file is the maximum size of your active log. If you have long running transactions or large messages (or both) then having small logs will give problems. See my other post (and/or the MQ manuals description of checkpoint) for more on the active log. |
|
| Back to top |
|
|
| mvic |
| Posted: Fri Jan 15, 2010 8:49 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| Vitor wrote: |
| Even if no media image has been taken and the queue manager's recovering all the way from the 1st file to the 3000th? |
This is a rare-to-never occurrence, and only happens if there has been a disk error leading to bad data loss from /var/mqm/qmgrs, so requiring the data to be read back from the media images in the linear logs.
| Quote: |
| I stand by my comment about disc space though. |
I agree with what you said about disk space. It sounds like the OP had no management of the log directory, which of course gives a problem when using linear logs.
UPDATE: Changed my question "when might this happen" into a statement of when I would expect it to happen.
Last edited by mvic on Fri Jan 15, 2010 8:54 am; edited 2 times in total |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jan 15, 2010 8:52 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mvic wrote: |
| Vitor wrote: |
| Even if no media image has been taken and the queue manager's recovering all the way from the 1st file to the 3000th? |
When might this happen? |
When the media image needs to be used? 
I suspect I'm missing your point here, and suspect it's as simple as I'm wrong about how the media image is wrong. This was the point of my question originally.....
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mvic |
| Posted: Fri Jan 15, 2010 8:59 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| Vitor wrote: |
| mvic wrote: |
| Vitor wrote: |
| Even if no media image has been taken and the queue manager's recovering all the way from the 1st file to the 3000th? |
When might this happen? |
When the media image needs to be used? |
Sorry I re-edited my post to answer my own question.
Media images are only used
- by rcrmqobj to recover an object damaged because of corruption - eg. disk error
- automatically when the queue manager is accessing an object on disk and finds it is damaged because of corruption - eg. disk error
This type of data corruption is a rare occurrence. Even after a queue manager ends badly (eg. crash) it should not need to do media replays, only replays of the data since checkpoint. |
|
| Back to top |
|
|
|
|
