ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » General IBM MQ Support » mq error log rotation

Post new topic  Reply to topic Goto page 1, 2  Next
 mq error log rotation « View previous topic :: View next topic » 
Author Message
crashdog
PostPosted: Thu Mar 15, 2018 4:33 am    Post subject: mq error log rotation Reply with quote

Voyager

Joined: 02 Apr 2017
Posts: 77

Hello,
I'm writing a process in c++ that "tails" the mq error log files AMQERR01.LOG. Basically that's straight forward and works.
The only thing that I'm a little unsure about is the rotation of the logfiles. From what I've seen on Unix systems, the file content gets piped from AMQERRO3.LOG to /dev/null. From AMQERR02.LOG to AMQERRO03.LOG and from AMQERR01.LOG to AMQERR02.LOG. The AMQERR01.LOG gets emptied but keeps it's inode number.
Now my main concern is that I would miss an entry before it's rotated. I don't want to parse the content in this process, it should only forward the messages to a queue.

So my question, how can I prevent the los/duplicate of an error log entry when the error log file gets rotated ?

ps: from what I've seen it actually works, but that could be just a coincidence.
Kind Regards,

Gerhard
_________________
You win again gravity !
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Mar 15, 2018 5:05 am    Post subject: Re: mq error log rotation Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9392
Location: US: west coast, almost. Otherwise, enroute.

crashdog wrote:
ps: from what I've seen it actually works, but that could be just a coincidence.

What is it?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
crashdog
PostPosted: Thu Mar 15, 2018 6:08 am    Post subject: Reply with quote

Voyager

Joined: 02 Apr 2017
Posts: 77

"it" is my process that reads the error log files continously and manages to correctly read the error file from start after it's been rotated. There are no missing or duplicate error log entries. But my fear is that it's a coincidence and that there might be some cases when my process might lose an entry or read one twice.

That would surely not happen if there's anything "hard" to rely on when the error logs rotate. Like for example the execution of the queue manager being postponed a few milliseconds during the rotation. Some kind of signal or... I don't know what's reliable to check for when the error log rotates.


Cheers,
Gerhard
_________________
You win again gravity !
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Mar 15, 2018 7:43 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9392
Location: US: west coast, almost. Otherwise, enroute.

So, your concern is that in the short time between 0001.LOG being renamed to 0002.LOG, and BEFORE a new 001.LOG is created, that some errors might NOT be recoded? Is that it?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
crashdog
PostPosted: Thu Mar 15, 2018 8:07 am    Post subject: Reply with quote

Voyager

Joined: 02 Apr 2017
Posts: 77

Basically yes. I know it sounds pedantic. But looking at timing issues in computers, sometimes milliseconds matter... when the event that the file has new information is promoted to my process, the error log might allready have rotated before the read completed and missed some important errors / information.

Quote:
So, your concern is that in the short time between 0001.LOG being renamed to 0002.LOG, and BEFORE a new 001.LOG is created, that some errors might NOT be recoded? Is that it?


actually I don't think the files are renamed... if they where, they would have different inode numbers.
_________________
You win again gravity !
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Mar 15, 2018 9:19 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9392
Location: US: west coast, almost. Otherwise, enroute.

crashdog wrote:
Basically yes. I know it sounds pedantic. But looking at timing issues in computers, sometimes milliseconds matter... when the event that the file has new information is promoted to my process, the error log might allready have rotated before the read completed and missed some important errors / information.

Quote:
So, your concern is that in the short time between 0001.LOG being renamed to 0002.LOG, and BEFORE a new 001.LOG is created, that some errors might NOT be recoded? Is that it?


actually I don't think the files are renamed... if they where, they would have different inode numbers.

I seem to recall that rename does not create a new inode (but does alter date/time stamp), however I will yield to a UNIX internals specialist.

I can't imagine that MQ internals would use the copy command to physically copy the entire error log files to accomplish this rotation.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
crashdog
PostPosted: Thu Mar 15, 2018 9:47 am    Post subject: Reply with quote

Voyager

Joined: 02 Apr 2017
Posts: 77

when you rename file 001.log to 002.log the 002.log will new have 001.log's inode number and 001.log needs a new inode number.

Code:

4500794 -rw-r--r--  1 xxx mqm      0 Mar 15 18:43 001.log
4500802 -rw-r--r--  1 xxx mqm      0 Mar 15 18:43 002.log
mv 002.log 003.log
mv 001.log 0002.log
 4500794 -rw-r--r--  1 xxx mqm      0 Mar 15 18:43 0002.log
 4500802 -rw-r--r--  1 xxx mqm      0 Mar 15 18:43 003.log


But in case of MQ I've seen that the inode numbers stay the same. Therefore my guess is that the file content is piped to the 002 and 003 logs.
_________________
You win again gravity !
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Thu Mar 15, 2018 3:32 pm    Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2491
Location: Melbourne, Australia

crashdog wrote:
Therefore my guess is that the file content is piped to the 002 and 003 logs.

File piping / copying does not sound very efficient, particularly if the error logs are large. I have always assumed that MQ Deletes 03, Renames 02 to 03, Renames 01 to 02, Writes to new 01.
_________________
Glenn
Back to top
View user's profile Send private message
tczielke
PostPosted: Fri Mar 16, 2018 8:28 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

It's really the inode that uniquely identifies a file in Unix. It has been a while since I last checked, but I do remember the behavior that the inodes do not change on the AMQERR0x.LOG queue manager error logs as the data is moved between the three files. This probably means that MQ is just programatically moving the data between these three files. So when it is time to move the data from AMQERR01.LOG to AMQERR02.LOG, MQ probably just programmatically clears the records from AMQERR02.LOG and then writes the data from AMQERR01.LOG into AMQERR02.LOG.

If your requirement is to capture all the messages that could get written to the queue manager error logs, you probably need a more sophisticated program that is sweeping through the different AMQERR0x.LOG files and checking file update timestamps to make sure it is not missing any data due to an error log rotation.
_________________
Working with MQ since 2010.


Last edited by tczielke on Fri Mar 16, 2018 9:21 am; edited 1 time in total
Back to top
View user's profile Send private message
bruce2359
PostPosted: Fri Mar 16, 2018 9:08 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9392
Location: US: west coast, almost. Otherwise, enroute.

gbaddeley wrote:
crashdog wrote:
Therefore my guess is that the file content is piped to the 002 and 003 logs.

File piping / copying does not sound very efficient, particularly if the error logs are large. I have always assumed that MQ Deletes 03, Renames 02 to 03, Renames 01 to 02, Writes to new 01.

My recollection is similar to that of gbaddeley.

I suspect that someone from IBM will be along to offer a more definitive explanation. I suspect, as well, that this is one of those 'how does x work' vs. 'exactly how does x work' instances.

Exactly how IBM internals accomplish any given task is proprietary, and may/will change over time and technology.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
zpat
PostPosted: Fri Mar 16, 2018 10:23 am    Post subject: Reply with quote

Jedi Council

Joined: 19 May 2001
Posts: 5849
Location: UK

They're probably all en-route to Vegas.....
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error.
Back to top
View user's profile Send private message
Vitor
PostPosted: Fri Mar 16, 2018 10:54 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

zpat wrote:
They're probably all en-route to Vegas.....


And any updates while they're there must be viewed with suspicion (unless accompanied by a negative breathalyzer reading!)
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
markt
PostPosted: Fri Mar 16, 2018 3:23 pm    Post subject: Reply with quote

Knight

Joined: 14 May 2002
Posts: 502

Yep ... sitting in LAX waiting for LAS after coming from SMF ... Just how many G&T can they serve on a one-hour ride.

There's some new options for diagnostic logging (I try not to say "error logging") in the just-available MQ V9.0.5 that may help with processing and parsing. Take a look at https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_9.0.0/com.ibm.mq.con.doc/q018795_.htm
Back to top
View user's profile Send private message
rekarm01
PostPosted: Fri Mar 16, 2018 8:28 pm    Post subject: Re: mq error log rotation Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 1415

When all else fails, check the Knowledge Center.

crashdog wrote:
The only thing that I'm a little unsure about is the rotation of the logfiles. From what I've seen on Unix systems, the file content gets piped from AMQERRO3.LOG to /dev/null ...

No piping. No /dev/null. But the behavior does depend on the version of MQ.

Prior to MQ v9.0.4, when AMQERR01.LOG gets full, the qmgr copies AMQERR02.LOG to AMQERR03.LOG, then copies AMQERR01.LOG to AMQERR02.LOG, then truncates AMQERR01.LOG. It is possible for one process to truncate a file, before another process can finish reading it.

As of MQ v9.0.4, when AMQERR01.LOG gets full, the qmgr removes AMQERR03.LOG, then renames AMQERR02.LOG to AMQERR03.LOG, then renames AMQERR01.LOG to AMQERR02.LOG, then creates a new AMQERR01.LOG. A process with an already open file handle can continue reading from an otherwise renamed (or even removed) file, but it would have to open a new file handle to read from a new file.

bruce2359 wrote:
I seem to recall that rename does not create a new inode (but does alter date/time stamp), however I will yield to a UNIX internals specialist.

The filename is a property of the directory entry, not the file. Changing the name in the directory entry alters the directory's timestamp, but has no effect whatsoever on the file that the directory entry points to.
Back to top
View user's profile Send private message
crashdog
PostPosted: Sat Mar 17, 2018 2:00 am    Post subject: Reply with quote

Voyager

Joined: 02 Apr 2017
Posts: 77

The knowledge center was the first place that I checked, before even starting this thread. https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.0.0/com.ibm.mq.tro.doc/q039570_.htm . But the KC does not describe anything on how the error log files are rotated.

By the way CD is out of scope in the organization that I currently work. It was decided to go with LTS.
At one point we will start using elastic for error log file statistics. Elastic uses filebeat to forward log information. I've read this artice on it : https://www.ibm.com/developerworks/community/blogs/messaging/entry/Storing_and_searching_MQ_logs_in_Elasticsearch?lang=en .
There would also have been logstash written i Java. But the idea at the moment is to use the c++ standard library (which is always installed on our systems) with statically linked QT functionallity for file handling. We don't want to install addtional frameworks such as java or go runtime.

I will try to setup an experimental queue manager in my homelab tomorrow and try to look further into the rotating part.
_________________
You win again gravity !
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » General IBM MQ Support » mq error log rotation
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.