| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Multi Instance Fail over does not occur if NFS Unmounted |
« View previous topic :: View next topic » |
| Author |
Message
|
| smeunier |
 Posted: Tue Jun 30, 2020 1:08 pm Post subject: Multi Instance Fail over does not occur if NFS Unmounted |
 |
|
Partisan
Joined: 19 Aug 2002
Posts: 305
Location: Green Mountains of Vermont
|
Environment: Linux Redhat V7.8
MQ Version: 9.1.0.5
NFS netapp V4
Scenario:
We have had the Multi-Instance configuration set for quite a while now and have run many different failover scenarios with success. Today we did a whatif the NFS mounts were unmounted. What we experienced was not favorable.
We unmounted NFS MQ mounts on both active and standby nodes. After 2 minutes, the active node was still showing active and the standby showing "No Status" on a dspmq -x No log message were recorded, nor any FDC dumps issued. Connectivity was loss is about all, but the MQ processes remained active.
We remounted the NFS MQ mounts and everything resumed like nothing happened. The active remained active, but the Standby was stopped, so we had to restart that.
While a failover would have been impossible, since NFS was not mounted, I would have expected something to show an error, indicating files could not be accessed. I would have expected the active node to fail with an FDC generated.
Does this seem normal? Has anyone tried this type of TEST? We were looking to simulate a NFS burp, to see what would happens, as a while ago, the netapp had a firmware update and took both active and standby down.
Are there any configuration that could be set that would have allowed MQ to notice this issue faster(or at all)? |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Tue Jun 30, 2020 2:17 pm Post subject: Re: Multi Instance Fail over does not occur if NFS Unmounted |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| smeunier wrote: |
| I would have expected something to show an error, indicating files could not be accessed. I would have expected the active node to fail with an FDC generated. |
Where did you look for errors?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Jun 30, 2020 4:28 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Did you try and generate any traffic?
A queue manager that's idle does not necessarily need to access its storage.
If you want to see the true behavior, put a persistent message on the queue manager while the storage has been removed (unmounted) and see what happens.
My guess is you will have to restart both the active and the standby once the storage has been restored.
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| smeunier |
| Posted: Wed Jul 01, 2020 4:19 am Post subject: Re: Multi Instance Fail over does not occur if NFS Unmounted |
|
|
Partisan
Joined: 19 Aug 2002
Posts: 305
Location: Green Mountains of Vermont
|
| Quote: |
Where did you look for errors? |
I looked in /var/mqm/errors and in the MQ error logs for for the Queue Manager which were NFS mounted, but obviously could not be written to. |
|
| Back to top |
|
|
| smeunier |
| Posted: Wed Jul 01, 2020 4:40 am Post subject: |
|
|
Partisan
Joined: 19 Aug 2002
Posts: 305
Location: Green Mountains of Vermont
|
| Quote: |
| Did you try and generate any traffic? |
We repeated the test and then tried some connectivity, without messages. This then cause the active Queue Manager to fail, generating some FDC files. So I get what your saying, about an idle QMGR. We also tried a test where the active was NFS unmounted, but the standby kept its mounts. Since the locks would have been released with the unmount on the active, I would have thought that the standby would have taken over, but it didn't.
These test are being performed to see if multi-instance is sturdy enough to withstand an outage of this type. I guess you cannot expect the heart to continue, when the blood supply has been removed.  |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
