| Author |
Message
|
| santnmq |
 Posted: Mon Feb 21, 2011 2:26 am Post subject: |
 |
|
Centurion
Joined: 11 Jan 2011
Posts: 125
|
| Quote: |
| circular logs are re-used when they no longer are needed for re-start. Whereas, linear logs are never re-used. |
I Agree with this ...
The confusion arises when we talk about the secondary log files in linear logging. In the simpe terms, if we are using linear logging, number of files is not constraint here. I got the below answer from info center for this.
----
Although secondary log files are defined for linear logging, they are not used in normal operation. If a situation arises when, probably due to long-lived transactions, it is not possible to free a file from the active pool because it might still be required for a restart, secondary files are formatted and added to the active log file pool
---- |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Mon Feb 21, 2011 2:43 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| santnmq wrote: |
| Quote: |
| circular logs are re-used when they no longer are needed for re-start. Whereas, linear logs are never re-used. |
I Agree with this ...
The confusion arises when we talk about the secondary log files in linear logging. In the simple terms, if we are using linear logging, number of files is not constraint here. I got the below answer from info center for this.
| Quote: |
----
Although secondary log files are defined for linear logging, they are not used in normal operation. If a situation arises when, probably due to long-lived transactions, it is not possible to free a file from the active pool because it might still be required for a restart, secondary files are formatted and added to the active log file pool
---- |
|
Not quite. Primary and secondary logs are not treated differently because they are linear or circular. The thing with linear of course is that a primary log is not overwritten.
Like I said primary and secondary logs are a way of managing your logspace. If you want the logspace to be reserved and show up as used in your OS, use primary only. If you want the logspace to be available but show as available (free) in the OS, use a maximum of secondary logs and a minimum of primary logs...
I believe that using more secondary logs gives you a better measure of how your logspace is being used and when it is time to extend it.
Also if you plan on using primary only in linear logging you better define the logspace as double of what you need so that you can archive easily as the used logspace will never drop below the number of primary logs... Remember that once the logspace is over 75% used the qmgr starts to "worry" and once the logspace is full it will stop processing UOWs.
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mvic |
| Posted: Mon Feb 21, 2011 3:11 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| fjb_saper wrote: |
| If you want the logspace to be available but show as available (free) in the OS, ... |
I am not arguing with the correctness of your statement, but the above is not a reasonable thing to want to achieve, IMHO. It does not help you as an MQ sys admin to have the OS think you need less space than you really do need. |
|
| Back to top |
|
|
| santnmq |
| Posted: Mon Feb 21, 2011 3:30 am Post subject: |
|
|
Centurion
Joined: 11 Jan 2011
Posts: 125
|
| fjb_saper, thanks for sharing this info to me... it's really useful. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Feb 21, 2011 7:28 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| mvic wrote: |
| fjb_saper wrote: |
| If you want the logspace to be available but show as available (free) in the OS, ... |
I am not arguing with the correctness of your statement, but the above is not a reasonable thing to want to achieve, IMHO. It does not help you as an MQ sys admin to have the OS think you need less space than you really do need. |
But that is why you allocate your file system size according to your calculated needs and monitor its usage.
Agreed you may not want to do this on Windows as you want to avoid releasing any of this space to a non MQ application. In Unix/Linux you determine the size of the mounts for /var/mqm and it's sub directories, so there is little risk of that happening... Monitoring the volume actually used under Unix/Linux gives you then a better appreciation as to when you need to grow it... especially if you have an application that is not consistent at committing its UOWs and they get freed once a day when the app is recycled...
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Mon Feb 21, 2011 7:50 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9471
Location: US: west coast, almost. Otherwise, enroute.
|
Your overall log space calculation needs to be based on your organization requirements; and for normal operation, the calculation must anticipate the use of both both primary and secondary logs.
Secondary logs are used when and if needed, and the space must remain available to WMQ.
Please (re)read the chapter on logs in the WMQ System Administration manual. It is pretty explicit.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| santnmq |
| Posted: Tue Feb 22, 2011 3:19 am Post subject: |
|
|
Centurion
Joined: 11 Jan 2011
Posts: 125
|
I think, we had a very positive discussion over this topic and this is going to be very benificial for all of us.
Thank you all for your comments!!! |
|
| Back to top |
|
|
| vikasmq |
| Posted: Tue Feb 22, 2011 12:07 pm Post subject: |
|
|
Newbie
Joined: 17 Aug 2010
Posts: 7
|
| Thanks for all the info, guys. The information is pretty useful. |
|
| Back to top |
|
|
| mqdogsbody |
| Posted: Tue May 08, 2012 5:44 am Post subject: |
|
|
Acolyte
Joined: 01 Jun 2010
Posts: 71
|
| fjb_saper wrote: |
| But that is why you allocate your file system size according to your calculated needs and monitor its usage. |
How can one monitor one's log usage?
- Do you just mean monitoring the file-system's free space?
- Or can one look for the creation of secondary log files? (From reading other messages I get the impression that these might be the best sign that a lot of log space is being used.) Do these have distinctive names?
- Is there an MQ utility or a RUNMQSC command that will say how much space is in use/free?
I gather that the UOWLOGTI and UOWLOGDA fields in the output of the MQSC "dis conn" command are the best way of determining who has the long running transaction. Any other/better techniques?
_________________
-- mqDB -- |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue May 08, 2012 6:04 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9471
Location: US: west coast, almost. Otherwise, enroute.
|
| mqdogsbody wrote: |
How can one monitor one's log usage?
- Do you just mean monitoring the file-system's free space?
- Or can one look for the creation of secondary log files? (From reading other messages I get the impression that these might be the best sign that a lot of log space is being used.) Do these have distinctive names?
- Is there an MQ utility or a RUNMQSC command that will say how much space is in use/free?
I gather that the UOWLOGTI and UOWLOGDA fields in the output of the MQSC "dis conn" command are the best way of determining who has the long running transaction. Any other/better techniques? |
You might do some research on you own. Begin by reading the relevant sections of the WMQ System Administration manual (or InfoCenter equivalent). How logs are named is well-documented.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| mqdogsbody |
| Posted: Fri May 11, 2012 6:36 am Post subject: |
|
|
Acolyte
Joined: 01 Jun 2010
Posts: 71
|
| bruce2359 wrote: |
| You might do some research on you own. |
Thanks for your help 
| bruce2359 wrote: |
| Begin by reading the relevant sections of the WMQ System Administration manual (or InfoCenter equivalent). |
Strangely enough I have, but looking for anything in the InfoCenter docs is like looking for a needle in several haystacks.
In any case, you guys, in your collective wisdom, might know things that aren't in the docs (or are not stated explicitly).
Am I right in assuming "Recovery and problem determination" is the chapter to concentrate on?
| bruce2359 wrote: |
| How logs are named is well-documented. |
That may be so, but I haven't yet found the relevant InfoCenter topic.
There's one on the directory structure that just says that the active "directory contains the log files numbered S0000000.LOG, S0000001.LOG, S0000002.LOG, and so on." Nothing about primaries and secondaries there. Since we use linear logging the total number of log files isn't much guide: it is always going to be increasing.
_________________
-- mqDB -- |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun May 13, 2012 5:53 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Using OS level tools you can count the number of logfiles in the active directory. If you run rcdmqimg once a day and use a support pack to archive the linear logs right afterwards, you should see how the total number of files grows over time, until it drops again...
Remember that it will not drop under the number of primary files...
So in Unix, if you have a dedicated file system, you can use a max of secondary logs and see the usage over time...
If you run support pack MS0L for archiving the linear logs, it will log the number of logs found in the active directory...
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqdogsbody |
| Posted: Mon May 14, 2012 2:16 am Post subject: |
|
|
Acolyte
Joined: 01 Jun 2010
Posts: 71
|
| fjb_saper wrote: |
| sing OS level tools you can count the number of logfiles in the active directory. |
That's what I was thinking of. But do I infer from what you go on to say (and what you don't say) that there is no way of distiguishing secondary from primary files. That makes things tricky.
I wonder if the "last modified" time might also provide a clue. But how would I distinguish a very busy QM from one with transactions backing up?
I suppose I could keep an eye out for the AMQ7467 messages or query the QM's RECLOG value to work out the number of logs which are live and so not be confused by ever increasing number of log files lying around. I imagine that whilst there are secondary files in existence RECLOG isn't going to change.
I guess I need to observe what actually happens, but I don't really have time to write a log-filling app and I am not going to be thanked if I cause a QM to hit "logs full". (Actually, I doubt anyone will notice on the QM I have in mind.)
But I am speculating. I was hoping to be told by those who know!
| fjb_saper wrote: |
| If you run rcdmqimg once a day and use a support pack to archive the linear logs right afterwards, you should see how the total number of files grows over time, until it drops again...I think |
We do that once a week, when we bounce our QM.
But would even once a day give us the granularity to detect a rogue client?
_________________
-- mqDB -- |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon May 14, 2012 4:03 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| mqdogsbody wrote: |
| fjb_saper wrote: |
| sing OS level tools you can count the number of logfiles in the active directory. |
That's what I was thinking of. But do I infer from what you go on to say (and what you don't say) that there is no way of distiguishing secondary from primary files. That makes things tricky.
I wonder if the "last modified" time might also provide a clue. But how would I distinguish a very busy QM from one with transactions backing up?
I suppose I could keep an eye out for the AMQ7467 messages or query the QM's RECLOG value to work out the number of logs which are live and so not be confused by ever increasing number of log files lying around. I imagine that whilst there are secondary files in existence RECLOG isn't going to change.
I guess I need to observe what actually happens, but I don't really have time to write a log-filling app and I am not going to be thanked if I cause a QM to hit "logs full". (Actually, I doubt anyone will notice on the QM I have in mind.)
But I am speculating. I was hoping to be told by those who know!
| fjb_saper wrote: |
| If you run rcdmqimg once a day and use a support pack to archive the linear logs right afterwards, you should see how the total number of files grows over time, until it drops again...I think |
We do that once a week, when we bounce our QM.
But would even once a day give us the granularity to detect a rogue client? |
Like I said, I run MS0L once and hour. This nicely shows the growth of the log files (secondaries) and the archiving after the run of rcdmqimg gets rid of the no longer used files...
Distinguishing between a busy qmgr and one where transactions are backing up is not the question. Transactions start backing up when you are lacking log space. This would mean that the number of active logs has reached your max (primaries + secondaries).
On a very busy qmgr on the other hand you would see the min log # move up as the checkpoints are reached.
What you will see however is if one of the applications has a long running transaction, as it will prevent the lowest log # to move up when doing a checkpoint. You would have to run rcdmqimg to be able to move up and sometimes even bounce the qmgr. (rogue behaving app).
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
