| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| One of 2 Full repositories being dropped periodically |
« View previous topic :: View next topic » |
| Author |
Message
|
| smeunier |
 Posted: Mon Nov 26, 2018 11:07 am Post subject: One of 2 Full repositories being dropped periodically |
 |
|
Partisan
Joined: 19 Aug 2002
Posts: 305
Location: Green Mountains of Vermont
|
Environment
AIX with OS 5.3 Running MQ V6.0.2.9
Before the rants begin with "Upgrade", this is a production environment running a high volume Pub/Sub environment with IBM Extended support for the backlevel version. It is what it is for now..........
At issue, there are three QMGRs in a cluster(A,B,C) Queue manager A and B are the full repositories. They are clustered to allow Broker (QMGR A) gateway access to spread the connection load (QMGR B and C). What has happened (Third instance), is that QMGR B has randomly stopped receiving CLUSTER updates for two SYSTEM.BROKER queues(CONTROL and STREAM). After the 60 days of no receiving update MQ does its cleanup and removes these queues from the QMGR B's repository. Then all hell breaks loose.
On a previous instance, after validating, that the Repository Manager process looks alive and well, channels are active, and the listener port is alive and well, we refreshed the Repository, which resolved the immediate issue. But here we are again with the same issue 30 days later........
Not sure what is happening, but I noticed today that on the other full repository (QMGR A), there was a message AMQ9418 Indicating that is was the only QMGR as a full repository. that when we started receiving message on QMGR B, that these two queues are no longer receiving updates and thus the 60 day messages.
- It looks like QMGR B is being dropped as a Full repository. Not sure how, or what the signal would be for that to occur. Both CLUSSDR and RCVR channels to/from that QMGR are fine. I'm at a loss here. Doing a REFRESH is not a friendly answer, as it is very disruptive to all the applications. So a couple questions:
- Is there a way other than a REFRESH on QMGR B to get it to resubscribe to CLUSTER updates? I have tried to update an attribute of one of the cluster queues to see if it would force a change/update, but to no avail. I see a status change on the SYSTEM.CLUSTER.COMMAND.QUEUE that it has put a message, but never see the same on the other side (QMGR B).
- Would killing the Cluster Repository be a good idea? It seems like this is tightly coupled to the QMGR and may cause issues with shared memory?!
- What else can I look for that would/might be the re-occurring issue?
Right now, my immediate goal, is how to get QMGR B Full repository back in sync again outside of doing a refresh.
Any yes, a move to MQ V9 is on the horizon, after hardware refreshes. |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Mon Nov 26, 2018 11:15 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
If you have an extended support agreement with IBM, what do they say?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Nov 26, 2018 1:14 pm Post subject: Re: One of 2 Full repositories being dropped periodically |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| smeunier wrote: |
| - Would killing the Cluster Repository be a good idea? It seems like this is tightly coupled to the QMGR and may cause issues with shared memory?! |
Heck no! I accept your situation can't get much worse, but killing hostages is a really bad idea. I doubt (due to the shared kernel resources you mention) the queue manager would be happy with that, and you might go from a queue manager that's not in the cluster to a queue manager that won't start without a lot of fiddling by the AIX sysadmins.
| smeunier wrote: |
| - What else can I look for |
Your copy of the extended support contract and IBM's phone number.
I agree with my worthy associate - it's PMR time and at last you can justify all the money that you've been spending on extended support all these years.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Nov 26, 2018 7:20 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
How many clusters are A, B and C in? (hope you say one)
QMGRA has manually defined Cluster Sender channel(s) to which QMGR(s)?
QMGRB has manually defined Cluster Sender channel(s) to which QMGR(s)?
QMGRC has manually defined Cluster Sender channel(s) to which QMGR(s)?
Does a Display of QMGRA confirm it is still a Full Repository for your cluster?
Does a Display of QMGRB confirm it is still a Full Repository for your cluster?
If you do a minor update to the clustered queue, like changing the Description Field (add a period), do both Full Repositories show that updated description field, only one, or neither?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mvic |
| Posted: Thu Dec 20, 2018 8:55 am Post subject: Re: One of 2 Full repositories being dropped periodically |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| smeunier wrote: |
| QMGR B has randomly stopped receiving CLUSTER updates for two SYSTEM.BROKER queues(CONTROL and STREAM) |
Why would Qmgr B ever want or receive updates for those qs? Each qmgr has its own queues of these names, and they are not clustered.
| Quote: |
| After the 60 days of no receiving update MQ does its cleanup and removes these queues from the QMGR B's repository. |
I assume you're still referring to queues named SYSTEM.*... I don't think the qmgr would remove its knowledge of these queues. They are defined locally to each QM.
| Quote: |
| Then all hell breaks loose. |
Presumably you see errors / bad return codes - what are they?
If you are supported at v6 then why not use the latest (6.0.2.12 I think) |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
