| Author |
Message
|
| ammx |
 Posted: Tue Sep 19, 2017 4:09 pm Post subject: filesystem data locks |
 |
|
Acolyte
Joined: 08 Sep 2017
Posts: 50
|
Hi
I am checking the qmgr error logs because my queue manager went down and I was searching for the cause of the problem. The error message is:
"AMQ7279: WebSphere MQ queue manager lost ownership of data lock.
The instance of the queue manager has lost ownership of a lock on its data in the filesystem due to transient failure. It was not able to reobtain the lock and will stop automatically to prevent the risk of data corruption".
I investigate a little bit about the filesystems that support reliable data locking and found that the NFS V4 is the one that supports file locking by a multi instance queue manager. I checked the filesystem where the qmgr is /var/mqm and the filesystem type is ext4, so I don't know if the filesystem supports this. |
|
| Back to top |
|
 |
| Anant.v |
| Posted: Tue Sep 19, 2017 8:09 pm Post subject: |
|
|
Apprentice
Joined: 26 Nov 2014
Posts: 40
Location: Malaysia
|
| What does amqmfsck tells ? |
|
| Back to top |
|
|
| ammx |
| Posted: Wed Sep 20, 2017 12:02 pm Post subject: |
|
|
Acolyte
Joined: 08 Sep 2017
Posts: 50
|
When I am amqmfsck /var/mqm it throws me the following message:
The tests on the directory completed successfully. |
|
| Back to top |
|
|
| JosephGramig |
| Posted: Wed Sep 20, 2017 12:33 pm Post subject: |
|
|
Grand Master
Joined: 09 Feb 2006
Posts: 1244
Location: Gold Coast of Florida, USA
|
Because you are using /var/mqm directory for your Qmgr, I would guess that you are not trying to use Multi Instance features. Right?
What version of the OS are you on?
What does dspmqinst say? |
|
| Back to top |
|
|
| ammx |
| Posted: Thu Sep 21, 2017 7:50 am Post subject: |
|
|
Acolyte
Joined: 08 Sep 2017
Posts: 50
|
It is a Linux RedHat 6, when I dspmqinst it shows the following:
InstName: Installation1
InstDesc:
Identifier: 1
InstPath: /opt/mqm
Version: 7.5.0.6
Primary: Yes
State: Available
so I guess I was wrong when I used the amqmfsck with the directory /var/mqm. But if I use the /opt directory it shows me an error message:
amqmfsck /opt/mqm
AMQ5667: Error executing system call 'open()' on file '/opt/mqm/amqmfsck.lck',
errno 13 ('Permission denied').
I know very little about Multi instance queue managers, so I am not sure how to troubleshoot this.
when I type dspmq to see the queue managers running on the server it only shows one active and another one which Ended unexpectedly, but there is no sign of another instance running as standby:
QMNAME(WCAPLQM3) STATUS(Running)
QMNAME(WCAPLQM1) STATUS(Ended unexpectedly)
But the error message makes me think that there is multi-instance configuration running in the server:
"The instance of queue manager WCAPLQM3 has lost ownership of a lock on its data in the filesystem due to a transient failure. It was not able to reobtain the lock and will stop automatically to prevent the risk of data corruption.
Check that another instance of the queue manager has become active. Restart this instance of the queue manager as a standby instance. If this problem recurs, it may indicate that the filesystem is not sufficiently reliable to support file locking by a multi-instance queue manager"
If I type mqsilist to list the brokers running I get the following:
BIP1295I: Integration node 'WCAPLBK3' is an active multi-instance or High Availability integration node that is running on queue manager 'WCAPLQM3'. |
|
| Back to top |
|
|
| JosephGramig |
| Posted: Thu Sep 21, 2017 8:25 am Post subject: |
|
|
Grand Master
Joined: 09 Feb 2006
Posts: 1244
Location: Gold Coast of Florida, USA
|
mqsilist always gives that answer if you created the Integration Node (broker) with the -e switch to put the Integration Node in a non default path (which is what you need to do when it is HA or MI).
dspmqinf WCAPLQM1
Should display where MQ expects the Qmgr to be located. If this is MI, then it will say where the MI NSFv4 mount point is located.
If this is a MI Qmgr, then you need to know where the "Standby" is/was running.
You need to know a lot. You should seek experienced help from some one on your team. If you have no resources like that at your company, IBM would be happy to sell you some services that will get you up to speed.
When working with MI Qmgr and you want to see the status, use:
dspmq -x |
|
| Back to top |
|
|
| ammx |
| Posted: Thu Sep 21, 2017 1:25 pm Post subject: |
|
|
Acolyte
Joined: 08 Sep 2017
Posts: 50
|
Hi
With this command I checked the path of the qmgr:
dspmqinf -o command WCAPLQM1, and it gave me this information:
addmqinf -s QueueManager -v Name=WCAPLQM1 -v Directory=WCAPLQM1 -v Prefix=/var/mqm -v DataPath=/HA1/MQ/qmgrs/WCAPLQM1
with this path I can check the shared filesystem type to see if it supports the data locks
df -P -T /HA1/MQ/qmgrs/WCAPLQM1 | tail -n +2 | awk '{print $2}'
and it throws me this result:
gpfs
It is a cluster filesystem and when I make the test with amqmfsck /HA1/MQ/qmgrs/WCAPLQM1 it says everything is fine.
"The tests on the directory completed successfully."
I need to perform other two tests in both servers, but I don't know in which other server is the other instance located
amqmfsck -c <directoryName> and amqmfsck -w <directoryName> |
|
| Back to top |
|
|
|
|
