ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » IBM MQ Installation/Configuration Support » MQ-Logging, deleting logfiles

Post new topic  Reply to topic
 MQ-Logging, deleting logfiles « View previous topic :: View next topic » 
Author Message
r3v4n
PostPosted: Sun Apr 11, 2021 11:13 pm    Post subject: MQ-Logging, deleting logfiles Reply with quote

Newbie

Joined: 26 Feb 2020
Posts: 5

Hey everyone,

first of all: I'm a real noob regarding MQ, so please be gentle

I've installed MQ

Code:
mqm@lvpaymqt01 [~]: dspmqver -a
Name:        IBM MQ
Version:     9.1.0.4
Level:       p910-004-191122
BuildType:   IKAP - (Production)
Platform:    IBM MQ for Linux (x86-64 platform)
Mode:        64-bit
O/S:         Linux 4.12.14-122.60-default
O/S Details: SUSE Linux Enterprise Server 12 SP5
InstName:    Installation1
InstDesc:
Primary:     Yes
InstPath:    /opt/mqm
DataPath:    /var/mqm
MaxCmdLevel: 910
LicenseType: Production


Code:

#*******************************************************************#
#* Module Name: qm.ini                                             *#
#* Type       : IBM MQ queue manager configuration file            *#
#  Function   : Define the configuration of a single queue manager *#
#*                                                                 *#
#*******************************************************************#
#* Notes      :                                                    *#
#* 1) This file defines the configuration of the queue manager     *#
#*                                                                 *#
#*******************************************************************#
ExitPath:
   ExitsDefaultPath=/var/mqm/exits
   ExitsDefaultPath64=/var/mqm/exits64
#*                                                                 *#
#*                                                                 *#
Log:
   LogPrimaryFiles=24
   LogSecondaryFiles=8
   LogFilePages=32432
   LogType=LINEAR
   LogBufferPages=0
   LogPath=/mqlogs/MQE1/
   LogWriteIntegrity=TripleWrite
#   LogManagement=Manual
   LogManagement=Automatic
Service:
   Name=AuthorizationService
   EntryPoints=14
ServiceComponent:
   Service=AuthorizationService
   Name=MQSeries.UNIX.auth.service
   Module=amqzfu
   ComponentDataSize=0
Channels:
   ChlauthEarlyAdopt=Y
   MaxChannels=65535
   MaxActiveChannels=9999
TCP:
   SndBuffSize=0
   RcvBuffSize=0
   RcvSndBuffSize=0
   RcvRcvBuffSize=0
   ClntSndBuffSize=0
   ClntRcvBuffSize=0
   SvrSndBuffSize=0
   SvrRcvBuffSize=0
SSL:
   AllowTLSV1=Y
   AllowWeakCipherSpec=TLS_RSA_WITH_AES_128_CBC_SHA


After a few days of use my log partition where all those S0000*.LOG files are held kept piling up.

I search the Internet and found out about LogManagement "Manual" and "Automatic".

I thought it would be a great idea to switch to "Automatic", because for a noob like me this sounded like "MQ will manage everything and automatically delete logs it doesn't need anymore!".

After I switched (as it can be seen in the qm.ini), I suddenly no have S0000*.LOG and R0000*.LOG files.

Stupid question: can I delete the R0000.LOG files? I've read something about those logs being for reuse or something? So are they still needed? In my AMQERR01.LOG it is said that only two specific S0000*.LOG files are need for restarting the QM.


Sorry for those maybe super stupid questions!
Back to top
View user's profile Send private message
crashdog
PostPosted: Mon Apr 12, 2021 5:33 am    Post subject: Reply with quote

Voyager

Joined: 02 Apr 2017
Posts: 77

Hello and welcome to the extraordinary world of message queuing.
There is an extensive library for IBM MQ called the "Knowledge Center" or short KC.
For some basics of MQ logging see here : https://www.ibm.com/docs/en/ibm-mq/9.1?topic=lost-types-logging
You definitively should not delete any log files called ..001.LOG MQ rotates log files when they reach a certain threshold of becoming full and MQ reuses some of the log files as you stated correctly when the queue manager is restarted or needs to be recovered.

If you don't need the recovery part of the log files you could create your queue manager with circular log rather then linear. Which means that the S00..*.LOG files will be reused after a while. But to learn this precisely there's no way around reading the documentation in the KC. So good luck with that.

Cheers,
Gerhard
_________________
You win again gravity !
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Tue Apr 13, 2021 9:30 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

you also need to have a look at the img* fields for the qmgr. They will determine how the qmgr treats the logs.
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
exerk
PostPosted: Tue Apr 13, 2021 9:41 am    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

crashdog wrote:
...If you don't need the recovery part of the log files you could create your queue manager with circular log rather then linear. Which means that the S00..*.LOG files will be reused after a while. But to learn this precisely there's no way around reading the documentation in the KC. So good luck with that.

Cheers,
Gerhard

Or if on V9.0.4, or above, migrate from linear to circular logging (migmqlog).
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
hughson
PostPosted: Tue Apr 13, 2021 3:33 pm    Post subject: Re: MQ-Logging, deleting logfiles Reply with quote

Padawan

Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand

r3v4n wrote:
Stupid question: can I delete the R0000.LOG files? I've read something about those logs being for reuse or something? So are they still needed?


The log extents named Rnnnnnnn.LOG will be reused by the queue manager over time. However, if you think you have too many of them and want to reduce them, you can use the following MQSC command:-

Code:
RESET QMGR TYPE(REDUCELOG) REDUCE(MAX)


r3v4n wrote:
In my AMQERR01.LOG it is said that only two specific S0000*.LOG files are need for restarting the QM.


Those are the oldest log files needed for restarting the queue manager, not the only ones needed.

The error messages you saw in AMQERR01.LOG would have looked something like this:-

Code:
AMQ7467: The oldest log file required to start queue manager MQG1 is S0000019.LOG.
AMQ7468: The oldest log file required to perform media recovery of queue manager MQG1 is S0000019.LOG.


Note the word 'oldest'.

Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software
Back to top
View user's profile Send private message Visit poster's website
r3v4n
PostPosted: Mon Aug 23, 2021 11:05 pm    Post subject: Reply with quote

Newbie

Joined: 26 Feb 2020
Posts: 5

exerk wrote:
crashdog wrote:
...If you don't need the recovery part of the log files you could create your queue manager with circular log rather then linear. Which means that the S00..*.LOG files will be reused after a while. But to learn this precisely there's no way around reading the documentation in the KC. So good luck with that.

Cheers,
Gerhard

Or if on V9.0.4, or above, migrate from linear to circular logging (migmqlog).


I've created a new QM (Version 9.1.5.0) with Circular Logging:

Code:

#*******************************************************************#
#* Module Name: qm.ini                                             *#
#* Type       : IBM MQ queue manager configuration file            *#
#  Function   : Define the configuration of a single queue manager *#
#*                                                                 *#
#*******************************************************************#
#* Notes      :                                                    *#
#* 1) This file defines the configuration of the queue manager     *#
#*                                                                 *#
#*******************************************************************#
ExitPath:
   ExitsDefaultPath=/var/mqm/exits
   ExitsDefaultPath64=/var/mqm/exits64
#*                                                                 *#
#*                                                                 *#
Log:
   LogPrimaryFiles=24
   LogSecondaryFiles=8
   LogFilePages=32432
   LogType=CIRCULAR
   LogBufferPages=0
   LogPath=/mqlogs/MQP1/
   LogWriteIntegrity=TripleWrite
   LogManagement=Manual


For my understanding Circular means, that

Quote:
With Circular logging, all the necessary log files are allocated up front. You never run out log files as they are used in a wrapping
fashion. [...]


Doesn't that mean, that my

Code:

mqm@lvpaymqp01 [/mqlogs/MQP1/active]: l
total 6097600
drwxrws--- 2 mqm mqm      4096 Aug 24 08:27 ./
drwxrws--- 3 mqm mqm      4096 Aug 13 09:24 ../
-rw-rw---- 1 mqm mqm 132849664 Aug 19 17:15 S0000000.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 19 17:58 S0000001.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 19 18:43 S0000002.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 19 21:01 S0000003.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 00:15 S0000004.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 06:34 S0000005.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 06:57 S0000006.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 07:25 S0000007.LOG
[...]


shouldn't grow but instead the S0*.LOG files should be reused?

Some of them seem to be reused (according to the timestamp) but my problem is, that per day 3+ new S0*.LOG files are created

What am I doing wrong?
Back to top
View user's profile Send private message
exerk
PostPosted: Tue Aug 24, 2021 1:50 am    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

You are doing nothing wrong that I can see.

It is my understanding that when you allocate the logs on a circular-logged queue manager, ALL the Primary logs are created in advance, therefore, expect to see ALL your Primary logs in the directory. I've just checked this on a Windows MQ V9.2.0 CD installation to verify, but don't have a UNIX-based machine at the moment to test it on that platform, but I would not expect the behaviour to be any different.

I'm not sure what you mean by "...per day 3+ new S0*.LOG files are created...", because 'see above'. Perhaps what you are seeing is that three or more logs are being filled each day (hence the last touch time/date changing), which should continue until log 0000024 is full, after which it will wrap around and start using log 0000000*

*CAVEAT: If you have a Unit-of-Work spanning ALL your Primary logs, then Secondary logs will be cut. If you manage to continue the span into ALL the Secondary logs, then expect stuff to be rolled back.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
r3v4n
PostPosted: Tue Aug 24, 2021 2:34 am    Post subject: Reply with quote

Newbie

Joined: 26 Feb 2020
Posts: 5

exerk wrote:
You are doing nothing wrong that I can see.

It is my understanding that when you allocate the logs on a circular-logged queue manager, ALL the Primary logs are created in advance, therefore, expect to see ALL your Primary logs in the directory. I've just checked this on a Windows MQ V9.2.0 CD installation to verify, but don't have a UNIX-based machine at the moment to test it on that platform, but I would not expect the behaviour to be any different.

I'm not sure what you mean by "...per day 3+ new S0*.LOG files are created...", because 'see above'. Perhaps what you are seeing is that three or more logs are being filled each day (hence the last touch time/date changing), which should continue until log 0000024 is full, after which it will wrap around and start using log 0000000*

*CAVEAT: If you have a Unit-of-Work spanning ALL your Primary logs, then Secondary logs will be cut. If you manage to continue the span into ALL the Secondary logs, then expect stuff to be rolled back.


Thank you very much for that quick response!

Quote:
I'm not sure what you mean by "...per day 3+ new S0*.LOG files are created...", because 'see above'. Perhaps what you are seeing is that three or more logs are being filled each day (hence the last touch time/date changing), which should continue until log 0000024 is full, after which it will wrap around and start using log 0000000*


It is my understanding as well, that there are S0000000.LOG, S0000001.LOG, S0000002.LOG, ... S0000024.LOG to be found in /mqlogs/MQP1/active

But sadly it looks like this:

Code:

mqm@lvpaymqp01 [/mqlogs/MQP1/active]: l /mqlogs/MQP1/active/
total 6486808
drwxrws--- 2 mqm mqm      4096 Aug 24 10:21 ./
drwxrws--- 3 mqm mqm      4096 Aug 13 09:24 ../
-rw-rw---- 1 mqm mqm 132849664 Aug 19 17:15 S0000000.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 19 17:58 S0000001.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 19 18:43 S0000002.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 19 21:01 S0000003.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 00:15 S0000004.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 06:34 S0000005.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 06:57 S0000006.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 07:25 S0000007.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 08:05 S0000008.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 13:12 S0000009.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 19:58 S0000010.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 23:45 S0000011.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 21 05:35 S0000012.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 23 17:12 S0000013.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 23 17:50 S0000014.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 23 18:31 S0000015.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 23 20:45 S0000016.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 04:42 S0000017.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 06:17 S0000018.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 06:48 S0000019.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 07:21 S0000020.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 07:53 S0000021.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 08:27 S0000022.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 09:11 S0000023.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 09:45 S0000024.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 10:21 S0000025.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 12:28 S0000026.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 19 21:01 S0000027.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 00:15 S0000028.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 06:34 S0000029.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 06:57 S0000030.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 07:25 S0000031.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 08:06 S0000032.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 13:12 S0000033.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 19:59 S0000034.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 20 23:45 S0000035.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 21 05:35 S0000036.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 23 17:12 S0000037.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 23 17:50 S0000038.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 23 18:31 S0000039.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 23 20:45 S0000040.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 04:42 S0000041.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 06:17 S0000042.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 06:48 S0000043.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 07:21 S0000044.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 07:54 S0000045.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 08:27 S0000046.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 09:11 S0000047.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 09:45 S0000048.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 10:21 S0000049.LOG


And

Code:

-rw-rw---- 1 mqm mqm 132849664 Aug 24 04:42 S0000041.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 06:17 S0000042.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 06:48 S0000043.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 07:21 S0000044.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 07:54 S0000045.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 08:27 S0000046.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 09:11 S0000047.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 09:45 S0000048.LOG
-rw-rw---- 1 mqm mqm 132849664 Aug 24 10:21 S0000049.LOG


alone where created today. And they keep coming.

I totally don't get it

/e: btw, that's some /mqdata/MQP1/errors/AMQERR01.LOG output:

Code:

----- amqalmp0.c : 626 --------------------------------------------------------
08/24/21 12:28:14 - Process(2524.10) User(mqm) Program(amqzmuc0)
                    Host(lvpaymqp01) Installation(Installation1)
                    VRMF(9.1.5.0) QMgr(MQP1)
                    Time(2021-08-24T10:28:14.116Z)
                    CommentInsert1(MQP1)
                    CommentInsert2(S0000026.LOG)

AMQ7467I: The oldest log file required to start queue manager MQP1 is
S0000026.LOG.

EXPLANATION:
The log file S0000026.LOG contains the oldest log record required to restart
the queue manager. Log records older than this may be required for media
recovery.
ACTION:
You can move log files older than S0000026.LOG to an archive medium to release
space in the log directory. If you move any of the log files required to
recreate objects from their media images, you will have to restore them to
recreate the objects. An older log file is one with a numerically smaller log
number (but allowing for log number wrapping at 9999999).
----- amqalmp0.c : 602 --------------------------------------------------------
08/24/21 12:28:14 - Process(2524.10) User(mqm) Program(amqzmuc0)
                    Host(lvpaymqp01) Installation(Installation1)
                    VRMF(9.1.5.0) QMgr(MQP1)
                    Time(2021-08-24T10:28:14.116Z)
                    CommentInsert1(MQP1)
                    CommentInsert2(S0000000.LOG)

AMQ7468I: The oldest log file required to perform media recovery of queue
manager MQP1 is S0000000.LOG.

EXPLANATION:
The log file S0000000.LOG contains the oldest log record required to recreate
any of the objects from their media images. Any log files prior to this will
not be accessed by media recovery operations.
ACTION:
Use this information together with the information in the most recent AMQ7467
message. Archivable log files are all those older than BOTH S0000000.LOG and
the log file mentioned in the AMQ7467 message.
----- amqalmp0.c : 626 --------------------------------------------------------
Back to top
View user's profile Send private message
exerk
PostPosted: Tue Aug 24, 2021 3:28 am    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

Something is wrong with your set-up, something I missed the first time around.

You stated that you created a circular-logged queue manager...
Code:
#*******************************************************************#
#* Module Name: qm.ini                                             *#
#* Type       : IBM MQ queue manager configuration file            *#
#  Function   : Define the configuration of a single queue manager *#
#*                                                                 *#
#*******************************************************************#
#* Notes      :                                                    *#
#* 1) This file defines the configuration of the queue manager     *#
#*                                                                 *#
#*******************************************************************#
ExitPath:
   ExitsDefaultPath=/var/mqm/exits
   ExitsDefaultPath64=/var/mqm/exits64
#*                                                                 *#
#*                                                                 *#
Log:
   LogPrimaryFiles=24
   LogSecondaryFiles=8
   LogFilePages=32432
   LogType=CIRCULAR
   LogBufferPages=0
   LogPath=/mqlogs/MQP1/
   LogWriteIntegrity=TripleWrite
   LogManagement=Manual

...and yet the parameter for LogManagement is included within the qm.ini file, an attribute which only appears if the queue manager is linear logged.

Your subsequent post, which shows new logs being cut and entries from the queue manager's error log showing linear-log print statements, bears this out as being a linear-logged queue manager.

Did you 'accidentally' define the queue manager as linear-logged, and then manually alter its qm.ini file to read CIRCULAR by any chance? If so, did you think that the queue manager would magically change from one type of logging to the other?

In an earlier post I informed you how to migrate from one type of logging to the other, and that is what you should use - or delete/redefine your queue manager with the correct logging type!
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
r3v4n
PostPosted: Tue Aug 24, 2021 6:02 am    Post subject: Reply with quote

Newbie

Joined: 26 Feb 2020
Posts: 5

exerk wrote:
Your subsequent post, which shows new logs being cut and entries from the queue manager's error log showing linear-log print statements, bears this out as being a linear-logged queue manager.

Did you 'accidentally' define the queue manager as linear-logged, and then manually alter its qm.ini file to read CIRCULAR by any chance? If so, did you think that the queue manager would magically change from one type of logging to the other?

In an earlier post I informed you how to migrate from one type of logging to the other, and that is what you should use - or delete/redefine your queue manager with the correct logging type!


Damn it, that's exactly what happened.

I really thought that it's enough to just change the LogType in the qm.ini

Learning the hard way.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Tue Aug 24, 2021 7:49 am    Post subject: Re: MQ-Logging, deleting logfiles Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

r3v4n wrote:
Hey everyone,

first of all: I'm a real noob regarding MQ, so please be gentle

Please, oh please, enroll in a hands-on MQ system administration class - such as https://www.ibm.com/training/course/WM156G
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » IBM MQ Installation/Configuration Support » MQ-Logging, deleting logfiles
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.