ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexGeneral IBM MQ SupportMulti Instance Fail over does not occur if NFS Unmounted

Post new topicReply to topic
Multi Instance Fail over does not occur if NFS Unmounted View previous topic :: View next topic
Author Message
smeunier
PostPosted: Tue Jun 30, 2020 1:08 pm Post subject: Multi Instance Fail over does not occur if NFS Unmounted Reply with quote

Partisan

Joined: 19 Aug 2002
Posts: 304

Environment: Linux Redhat V7.8
MQ Version: 9.1.0.5

NFS netapp V4

Scenario:

We have had the Multi-Instance configuration set for quite a while now and have run many different failover scenarios with success. Today we did a whatif the NFS mounts were unmounted. What we experienced was not favorable.

We unmounted NFS MQ mounts on both active and standby nodes. After 2 minutes, the active node was still showing active and the standby showing "No Status" on a dspmq -x No log message were recorded, nor any FDC dumps issued. Connectivity was loss is about all, but the MQ processes remained active.

We remounted the NFS MQ mounts and everything resumed like nothing happened. The active remained active, but the Standby was stopped, so we had to restart that.

While a failover would have been impossible, since NFS was not mounted, I would have expected something to show an error, indicating files could not be accessed. I would have expected the active node to fail with an FDC generated.

Does this seem normal? Has anyone tried this type of TEST? We were looking to simulate a NFS burp, to see what would happens, as a while ago, the netapp had a firmware update and took both active and standby down.

Are there any configuration that could be set that would have allowed MQ to notice this issue faster(or at all)?
Back to top
View user's profile Send private message
bruce2359
PostPosted: Tue Jun 30, 2020 2:17 pm Post subject: Re: Multi Instance Fail over does not occur if NFS Unmounted Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8771
Location: US: west coast, almost. Otherwise, enroute.

smeunier wrote:
I would have expected something to show an error, indicating files could not be accessed. I would have expected the active node to fail with an FDC generated.

Where did you look for errors?
_________________
My life flows on in endless song;
How can I keep from singing?
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Tue Jun 30, 2020 4:28 pm Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20385
Location: LI,NY

Did you try and generate any traffic?
A queue manager that's idle does not necessarily need to access its storage.
If you want to see the true behavior, put a persistent message on the queue manager while the storage has been removed (unmounted) and see what happens.
My guess is you will have to restart both the active and the standby once the storage has been restored.
Enjoy
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
smeunier
PostPosted: Wed Jul 01, 2020 4:19 am Post subject: Re: Multi Instance Fail over does not occur if NFS Unmounted Reply with quote

Partisan

Joined: 19 Aug 2002
Posts: 304

Quote:

Where did you look for errors?


I looked in /var/mqm/errors and in the MQ error logs for for the Queue Manager which were NFS mounted, but obviously could not be written to.
Back to top
View user's profile Send private message
smeunier
PostPosted: Wed Jul 01, 2020 4:40 am Post subject: Reply with quote

Partisan

Joined: 19 Aug 2002
Posts: 304

Quote:
Did you try and generate any traffic?


We repeated the test and then tried some connectivity, without messages. This then cause the active Queue Manager to fail, generating some FDC files. So I get what your saying, about an idle QMGR. We also tried a test where the active was NFS unmounted, but the standby kept its mounts. Since the locks would have been released with the unmount on the active, I would have thought that the standby would have taken over, but it didn't.

These test are being performed to see if multi-instance is sturdy enough to withstand an outage of this type. I guess you cannot expect the heart to continue, when the blood supply has been removed.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Page 1 of 1

MQSeries.net Forum IndexGeneral IBM MQ SupportMulti Instance Fail over does not occur if NFS Unmounted
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.