MQSeries.net :: View topic

tkaravind · Posted: Thu Feb 14, 2008 7:07 am Post subject:

Hi Everyone,

We had a tricky cluster problem recently in our Production setup.

A Production cluster partial repository queue manager M2P1 ( z/OS - WMQ V 5.3.1) recently got several error messages of the format:

CSQX430E MQM2P1 CSQXREPO Unexpected queue manager repository command,
cluster CLUSP1,
channel TO.M2P1,
sender M1P1.BC9896318D64E200
CSQX428E MQM2P1 CSQXREPO Unexpected queue or query repository command,
cluster CLUSP1,
channel TO.M2P1,
sender M1P1.BC9896318D64E200
CSQX428E MQM2P1 CSQXREPO Unexpected queue or query repository command,
cluster CLUSP1,
channel TO.M2P1,
sender MCP1.BBF1426A4F432700
CSQX428E MQM2P1 CSQXREPO Unexpected queue or query repository command,
cluster CLUSP1,
channel TO.M2P1,
sender REPOSP1_2004-06-14_08.36.12

This showed that M2P1 is wrongly seen as FULL REPOSITORY by atleast two other queue managers M1P1 & REPOSP1.

REPOSP1 is a FULL REPOSITORY queue manager (Linux - WMQ V 5.3)
and M1P1 another partial repository queue manager M2P1 ( z/OS - WMQ V 5.3.1) .

Upon further investigation we found that on REPOSP1 there was a cluster channel wrongly defined ( TO.M2P1 ) which seem to have confused the
FULL REPOSITORY as to the status of the M2P1 in the cluster. ( M2P1 is defined only as a normal queue manager)

We have since deleted that channel. But not sure if this delete alone has successfully corrected the M2P1 status.

As of now dis clusqmgr(M2P1) returns QMTYPE(REPOS) on just REPOSP1 & M1P1.

[ Interestingly other queue managers in the same cluster that subscribe to M2P1 - including a second full REPOS , REPOSP3, see M2P1 as QMTYPE(NORMAL) ].

This situation has caused heavy message traffic to flow from REPOSP1 to M2P1 unnecessarily and CPU utilization has gone high, significantly delaying other processing. It also causes the errors listed above as M2P1 cannot accept such messages meant for a Full REPOS.

I have the following questions :

a) Even with a wrong cluster channel TO.M2P1 defined on REPOSP1 why are only 2 queue managers ( M1P1 & REPOSP1 ) seeing M2P1 as a full REPOS when others see it as normal. Would REPOSP1 not send the information to other queue managers in the cluster, many of which have a subscription to M2P1 including the second full REPOS, REPOSP3 ?

b) How do we clean this up and return M2P1 to its normal status even with M1P1 & REPOSP1

i.e dis clusqmgr on REPOSP1 & M1P1 should also return QMTYPE(NORMAL)

Your help much appreciated.

Thanks & Regards,
Aravind