| Author |
Message
|
| nimconsult |
 Posted: Mon Feb 24, 2003 11:58 pm Post subject: [Solved] Queue manager vanished after shutdown! |
 |
|
Master
Joined: 22 May 2002
Posts: 268
Location: NIMCONSULT - Belgium
|
I am sure in advance that most of you will not believe me.
The story is simple:
- pSeries 630 with AIX 5.1 installed.
- MQ Series 5.2 CSD05 installed.
- Two queue managers created, with circular logging, up and running.
Friday 21th
- For a reason external to MQ Series, we decide to perform a shutdown/restart of the machine.
- Login as root, "shutdown -r"
- The system shuts down, then restart, up and running again after 5-10 minutes.
- I logon with mqm, and there I see... that the two queue managers have disappeared! 
Let me explain what I mean by disappeared:
- MQ Series still sees the queue manager (there is still the entry in /var/mqm/mqs.ini).
- The directory /var/mqm/qmgrs/xxx exists, but is entirely empty!
- 100% sure that the queue manager was there just before the shutdown.
- After a first investigation, we found that the scripts neccesary to stop the queue manager at shutdown were not called. The machines were installed recently and the system guy forgot to call the start/stop scripts for boot/shutdown. What this means is that the queue managers were "killed" instead of being ended properly.
In my opinion the fact that the queue managers were not stopped properly simply means that a recover would be performed at restart. In no way did I expect the queue managers to disappear.
Has anybody ever met such a strange case?
(As I told you few of you will believe me, but believe me   )
Thank you in advance for any input you might give me.
_________________
Nicolas Maréchal
Senior Architect - Partner
NIMCONSULT Software Architecture Services (Belgium)
http://www.nimconsult.be |
|
| Back to top |
|
 |
| bduncan |
| Posted: Tue Feb 25, 2003 1:14 pm Post subject: |
|
|
Padawan
Joined: 11 Apr 2001
Posts: 1554
Location: Silicon Valley
|
YOU'RE LYING!!!! 
Actually, that is quite a weird experience. In the past, I have definitely used 'shutdown -r' while a queue manager was running (and there was no call to endmqm), and you are correct, when the machine came back up, and we ran strmqm, it took quite a while because a recovery was done since it wasn't shut down properly. But I've never seen the queue manager just go away. I'm sure this is too obvious to even ask, but is there any chance that the /var/mqm/qmgrs/xxx directory was nfs mounted? A look at /etc/fstab should confirm this... Otherwise I have no idea what might have caused it...
_________________
Brandon Duncan
IBM Certified MQSeries Specialist
MQSeries.net forum moderator |
|
| Back to top |
|
|
| nimconsult |
| Posted: Tue Feb 25, 2003 11:36 pm Post subject: |
|
|
Master
Joined: 22 May 2002
Posts: 268
Location: NIMCONSULT - Belgium
|
I KNEW IN ADVANCE THAT YOU WOULD NOT BELIEVE ME 
_________________
Nicolas Maréchal
Senior Architect - Partner
NIMCONSULT Software Architecture Services (Belgium)
http://www.nimconsult.be |
|
| Back to top |
|
|
| nimconsult |
| Posted: Wed Feb 26, 2003 12:42 am Post subject: |
|
|
Master
Joined: 22 May 2002
Posts: 268
Location: NIMCONSULT - Belgium
|
_________________
Nicolas Maréchal
Senior Architect - Partner
NIMCONSULT Software Architecture Services (Belgium)
http://www.nimconsult.be |
|
| Back to top |
|
|
| nimconsult |
| Posted: Wed Feb 26, 2003 12:49 am Post subject: |
|
|
Master
Joined: 22 May 2002
Posts: 268
Location: NIMCONSULT - Belgium
|
We found the solution of the problem
The solution is stupid. As I told you I did not install this machine, and I did not realize that individual mount points were created per queue manager (/var/mqm/qmgrs/xxx) 
MANY THANKS FOR YOUR HELP BRANDON
It is thanks to your comment on NFS that I investigated more and discovered the mount points. They are not NFS, but local journaled file systems, but at least I discovered the problem. 
I will have to find now why these filesystems were not mounted at startup
Thank you again,
_________________
Nicolas Maréchal
Senior Architect - Partner
NIMCONSULT Software Architecture Services (Belgium)
http://www.nimconsult.be |
|
| Back to top |
|
|
| mqonnet |
| Posted: Wed Feb 26, 2003 6:02 am Post subject: |
|
|
Grand Master
Joined: 18 Feb 2002
Posts: 1114
Location: Boston, Ma, Usa.
|
I also have encountered the error of "journaled file systems" not being mounted on startup. Am still investigating this. But if you get any near to this, please do update this post. Even though my problem is on Linux. But i guess if you get around it, i might have a clue to go ahead.
From my investigation i found that this happens when your MBR is modified in the event you have more than one operating system, say linux and 2k in my case, on the same hard drive and somehow you messed it up. Which is what happenned in my case, :). Grub failed to get started and hence all this trouble.
Cheers
Kumar
_________________
IBM Certified WebSphere MQ V5.3 Developer
IBM Certified WebSphere MQ V5.3 Solution Designer
IBM Certified WebSphere MQ V5.3 System Administrator |
|
| Back to top |
|
|
| nimconsult |
| Posted: Wed Feb 26, 2003 11:57 pm Post subject: |
|
|
Master
Joined: 22 May 2002
Posts: 268
Location: NIMCONSULT - Belgium
|
I found the reason why the file systems (JFS) were not mounted.
Note to Kumar: This is maybe specific to AIX and I am not sure that the reason is applicable to Linux.
The reason is linked to the logical volume manager of AIX. The file system is part of a volume group (or more exactly is part of a logical volume in a volume group).
There is an option for a volume group to enable/disable automatic activation at boot time. The automatic activation was disabled, so no file system part of this volume group was mounted.
At least, this problem was good for my education . Thank you again for your support.
Nicolas
_________________
Nicolas Maréchal
Senior Architect - Partner
NIMCONSULT Software Architecture Services (Belgium)
http://www.nimconsult.be |
|
| Back to top |
|
|
| mqonnet |
| Posted: Thu Feb 27, 2003 5:42 am Post subject: |
|
|
Grand Master
Joined: 18 Feb 2002
Posts: 1114
Location: Boston, Ma, Usa.
|
Thanks nicolas for sharing the info. I shall try to take this just as a clue and carry on further.
Cheers.
Kumar
_________________
IBM Certified WebSphere MQ V5.3 Developer
IBM Certified WebSphere MQ V5.3 Solution Designer
IBM Certified WebSphere MQ V5.3 System Administrator |
|
| Back to top |
|
|
|
|
