| Author |
Message
|
| mmorri8305 |
 Posted: Mon Jan 30, 2023 11:54 am Post subject: MQ v9 linear logging strange behavior |
 |
|
Newbie
Joined: 24 Sep 2015
Posts: 9
|
| I have been trying to set up automatic log clearing feature that has come available in MQ V9. Currently we are at version 9.2.0.6. What we are seeing is one of the log files disappears or doesn't show up in the active folder for a couple of days then it reappears as reusable. For example S0015789 does not show up in ls listing but 2 days later R0015789 appears. Is this normal behavior? I am trying to get comfortable using the utility before I roll back my custom in house script. I would rather use something IBM created if I am sure it works. Thanks for your help in advance. |
|
| Back to top |
|
 |
| hughson |
| Posted: Mon Jan 30, 2023 5:00 pm Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1959
Location: Bay of Plenty, New Zealand
|
Could you describe a little more about how you have set up your queue manager to control automatic management of linear log extents.
For example, there are two modes, automatic mode and archive mode. This is controlled by the qm.ini attribute LogManagement in the Log: stanza. Perhaps you can show us your qm.ini Log: stanza contents?
You also don't mention what platform you are using which might be useful to know?
As you are no doubt aware, the log extents that are available for reuse are renamed from S00nnnnn.LOG to R00nnnnn.LOG. You mention that a log extent called S0015789.LOG does not show up for 2 days. Could you tell us what tool or command you are using to view the active folder when you see this disappearing act?
Could you show the output of DISPLAY QMSTATUS ALL at the time when this disappearance happens?
You also mention a utility that you are trying to get comfortable with. Could you tell us what the utility is?
Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| mmorri8305 |
| Posted: Tue Jan 31, 2023 6:35 am Post subject: |
|
|
Newbie
Joined: 24 Sep 2015
Posts: 9
|
Hello Morag,
You are so helpful. I appreciate your help. We are on a Red Hat Linux platform. When I do a long listing of the file contents of the active folder, I don't see the folder for a couple days. Then I see it again, but it is marked for reuse. We are using automatic log management.
Log:
LogPrimaryFiles=9
LogSecondaryFiles=6
LogFilePages=8192
LogType=LINEAR
LogBufferPages=0
LogPath=/var/mqm/log/USLNTST01!MQM/
LogWriteIntegrity=TripleWrite
LogManagement=Automatic
By utility I mean Automatic Log Management.
In production I am using my own script to manage the deletion of older files. I would like to use Automatic Log Management in production if I can ensure it is working correctly.
Thanks, Mike |
|
| Back to top |
|
|
| hughson |
| Posted: Tue Jan 31, 2023 8:01 pm Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1959
Location: Bay of Plenty, New Zealand
|
Thank you for expanding on the details. It is always the case that the "devil is in the details" so it very hard to even begin thinking about problems like these without knowing the details.
In my previous post I asked:-
| hughson wrote: |
| Could you tell us what tool or command you are using to view the active folder when you see this disappearing act? |
To which I believe this is your answer?
| mmorri8305 wrote: |
| When I do a long listing of the file contents of the active folder, I don't see the folder for a couple days. Then I see it again, but it is marked for reuse. |
I'm afraid I still don't know what it is you are using to view the active folder. You have called it a "long listing". Is there are command or a UI that you are using to do this? I do apologise if this is obvious to you, but it is not obvious to me. I could interpret this as meaning you used the following command:-
but I would rather you told me exactly what you are using. The reason I am pressing this point is because I fully expect that the log extent file IS still there. It is just renamed from S00nnnnn.LOG to R00nnnnn.LOG once it is available for re-use. However there may be something about the file that - whatever it is you are using to look at it with - means it is not visible to that tool or command. Knowing EXACTLY what you are using to view the contents of the active folder could therefore be quite important to getting to the bottom of this oddity.
I mean, it might be as simple as to tell you to use ls -al instead of ls -l, who knows! Or asking you whether it shows up using find?
Also, when you say "I don't see the folder for a couple days", you just mean one file in the folder as you earlier described right? The whole folder doesn't disappear does it?
Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Feb 01, 2023 9:24 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
And are you always using the same user (mqm) ?
The folder disappearing could be due to missing permissions for the user. 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Andyh |
| Posted: Thu Feb 02, 2023 4:46 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 239
|
If my memory is correct then the R suffixed file only comes into existence through the renaming of the correspondingly named S suffixed file using the unix rename subroutine, and so the appearance of the R suffixed file seems to imply the existence of the S prefixed link to the file. I wouldn't expect the permissions to be changed at the time of the rename and hence it would be a bit of a surprise that you could ever see one and not the other.
This code has been around for a few years now and so it should be pretty stable. The most likely cause of any issues in this area would be if the user was to enable MQ's log management without completely removing their own log management code. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Feb 02, 2023 2:43 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
I must have missed the memo ...
Prefix used to indicate something that occurred at the left; suffix at the right. Pre=before, suffix=after
S0015789.LOG prefix is 'S'.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Andyh |
| Posted: Fri Feb 03, 2023 2:00 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 239
|
Doh!!! I've obviously been away for too long  |
|
| Back to top |
|
|
| mmorri8305 |
| Posted: Fri Feb 03, 2023 9:04 am Post subject: |
|
|
Newbie
Joined: 24 Sep 2015
Posts: 9
|
| Yes, this is just the Linux command to list the files. I actually use ls -altr so that they sorted, and I list the hidden files. The one file is not appearing in the list for a couple of days but reappears with R00XXXX instead of S00XXXX. Then the next file sequest number will disappear for a couple of days. It follows the same pattern then reappears. I am logged in as the mqm user. |
|
| Back to top |
|
|
| Andyh |
| Posted: Sat Feb 04, 2023 5:35 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 239
|
Sorry if this seems a bit of a basic suggestion, but is it possible the file simply isn't appearing in the position in the list where you expect it?
The recovery log is very frequently updated (every time the log is forced) and where it's possible the file will be opened in such a way where the file meta data (specifically modification time) doesn't get updated on each physical write. This sometimes results in log files not appearing in numerical order.
My earlier update suggested that the R file is created from a simple rename of the S file and so the existence of the R file implies the previous existence of the S file. MQ itself doesn't move these files and so that seems to suggest that either the file is in the directory all along, or that some user process is manipulating these files (as previously questioned). |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Feb 05, 2023 8:01 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Have you tried sorting your output skipping the first file letter?
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| hughson |
| Posted: Sun Feb 05, 2023 6:36 pm Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1959
Location: Bay of Plenty, New Zealand
|
| mmorri8305 wrote: |
| I actually use ls -altr so that they sorted, and I list the hidden files. The one file is not appearing in the list for a couple of days but reappears with R00XXXX instead of S00XXXX. Then the next file sequest number will disappear for a couple of days. |
Thank you for telling us the command you are using.
ls -t is requesting that you sort the listing by modification time. But you are complaining that the list is not shown to you in name order. I think Andy's suggestion may be quite right.
| Andyh wrote: |
Sorry if this seems a bit of a basic suggestion, but is it possible the file simply isn't appearing in the position in the list where you expect it?
The recovery log is very frequently updated (every time the log is forced) and where it's possible the file will be opened in such a way where the file meta data (specifically modification time) doesn't get updated on each physical write. This sometimes results in log files not appearing in numerical order.
My earlier update suggested that the R file is created from a simple rename of the S file and so the existence of the R file implies the previous existence of the S file. MQ itself doesn't move these files and so that seems to suggest that either the file is in the directory all along, or that some user process is manipulating these files (as previously questioned). |
Can you try again without the -t parameter on your ls command and see how you get on?
Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| mmorri8305 |
| Posted: Thu Feb 09, 2023 9:57 am Post subject: MQ v9 linear logging strange behavior |
|
|
Newbie
Joined: 24 Sep 2015
Posts: 9
|
| We have been monitoring the situation closely for a number of weeks now and finally there are no files missing. We have not changed anything on our end. The only thing that could have had an effect is we restarted our environment because we had to take the server down for patching. But we had restarted the environment since we made the settings so even that does not make sense. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Thu Feb 09, 2023 3:17 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
It might occur again.... Does your monitoring script record the output of "ls -l" on a regular basis? This will provide solid evidence for file sequence numbers apparently disappearing and reappearing.
_________________
Glenn |
|
| Back to top |
|
|
| mmorri8305 |
| Posted: Fri Feb 10, 2023 7:09 am Post subject: |
|
|
Newbie
Joined: 24 Sep 2015
Posts: 9
|
| gbaddeley wrote: |
| It might occur again.... Does your monitoring script record the output of "ls -l" on a regular basis? This will provide solid evidence for file sequence numbers apparently disappearing and reappearing. |
Actually, we are just monitoring manually. |
|
| Back to top |
|
|
|
|
