ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » Clustering » AMQ9456 Question

Post new topic  Reply to topic
 AMQ9456 Question « View previous topic :: View next topic » 
Author Message
tczielke
PostPosted: Mon Oct 06, 2014 6:19 am    Post subject: AMQ9456 Question Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

FYI - This was also cross-posted on the MQ LISTSERV.

I will probably open a PMR on this clustering behavior I am encountering, but I was curious if anyone on the LISTSERV has any insight to what I am seeing.

We have a qmgr/cluster set up like the following:

(QMGR1 on Server1 in CLUS1) -> (QMGR2 on Server2 in CLUS1 and CLUS2) -> (QMGR3 on Server3 in CLUS2)

QMGR1 can not be in CLUS2 and QMGR3 can not be in CLUS1 due to network rules.

Apps on Server1 have a requirement where they need to be able to PUT to a clustered queue Q1 in CLUS2 on QMGR3.

We have accomplished that by setting up the following:

1. A qalias for Q1 that is hosted on QMGR2 and is in CLUS1.
2. A local queue of Q1 on QMGR3 that is in CLUS2.

When an application PUTs to Q1 on QMGR1, it is sent to QMGR2 because of its qalias for Q1 in CLUS1. When the message gets to QMGR2, the queue manager sees the queue exists on QMGR3 in CLUS2, and sends the message to QMGR3.

However, when we review our queue manager error logs, we see reports like the following on QMGR1.


10/02/2014 11:24:36 PM - Process(13507.1) User(mqm) Program(amqrrmfa)
Host(Server1) Installation(Installation1)
VRMF(7.5.0.2) QMgr(QMGR1)

AMQ9456: Update not received for queue Q1, queue manager
QMGR2 from full repository for cluster CLUS1.

EXPLANATION:
The repository manager detected a cluster queue that had been used sometime in
the last 30 days for which updated information should have been sent from a
full repository. However, this has not occurred.

The repository manager will keep the information about this queue for a further
60 days from when the error first occurred.
ACTION:
If the queue is still required, check that:
1) The cluster channels to and from the full repository and the queue manager
that hosts the queue, are able to run.
2) The repository managers running on these queue managers have not ended
abnormally.


This looks like a potential clustering issue to me, but would this be expected behavior for this cluster set up? I would assume the full repositories on CLUS1 would be publishing information for the clustered qalias Q1 to QMGR1. Also those action items 1 and 2 in the AMQ9456 check out fine for QMGR1, QMGR2, and the FRs.

Please let me know if anyone has seen this behavior, or has any insight to what we are seeing.
Back to top
View user's profile Send private message
Vitor
PostPosted: Mon Oct 06, 2014 6:25 am    Post subject: Re: AMQ9456 Question Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

tczielke wrote:
Please let me know if anyone has seen this behavior, or has any insight to what we are seeing.


Why did you decide to alias at the queue level rather than set up a queue manager alias for QMGR1 & QMGR3 on QMGR2 that could service any queue falling into this (fairly common) scenario?

Can we also assume that both CLUS1 & CLUS2 are working normally, the channels are running, the correct queue managers show up in the CLUSQMGR display and you've made all the usual checks before posting?
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
tczielke
PostPosted: Mon Oct 06, 2014 7:11 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

Hi Vitor,

Most of our applications are coded to open a queue like Q1 with a blank ObjectQmgrName. So to set this up from an MQ standpoint and not require an app change (like having the app explicitly reference a qmgr alias in the ObjectQmgrName), we advertise the clustered queue alias as described on QMGR2.

I have verified that the QMGR1 and CLUS1 FR can talk to each other, the Q1 appears correctly in both repositories, and the repository managers are running on these queue managers. The same holds true for CLUS2.
Back to top
View user's profile Send private message
Vitor
PostPosted: Mon Oct 06, 2014 7:23 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

tczielke wrote:
Most of our applications are coded to open a queue like Q1 with a blank ObjectQmgrName. So to set this up from an MQ standpoint and not require an app change (like having the app explicitly reference a qmgr alias in the ObjectQmgrName), we advertise the clustered queue alias as described on QMGR2.


Fair point.

tczielke wrote:
I have verified that the QMGR1 and CLUS1 FR can talk to each other, the Q1 appears correctly in both repositories, and the repository managers are running on these queue managers. The same holds true for CLUS2.


Who are the FRs for these clusters? Do all the queue managers (not just Q1) show in the cluster object listings?

This smells like a broken cluster(s) at first thought.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
tczielke
PostPosted: Mon Oct 06, 2014 9:09 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

There are 2 FRs for each cluster. They run on their own distributed servers, and are at 7.5.0.3. The DIS CLUSQMGR(*) and QC(*) output looks good for the qmgrs mentioned in CLUS1.

For some more context, we saw a very similar issue when our FRs were at 7.5.0.2, and opened a PMR. After doing data collection on the clusters and queue managers, IBM said they saw nothing wrong with the clusters, but it could be due to an APAR at 7.5.0.2, that was corrected at 7.5.0.3. It was too late in our maintenance cycle to apply 7.5.0.3 fix packs to all our queue managers, but the recommendation was that just applying 7.5.0.3 to the FRs could resolve the issue. We did that and the messages went away. But now they are back. I looked at that PMR, and it is for different applications where cluster local queue information (i.e. that resides on QMGR10) was not being published from the FRs to the PRs (i.e. that resides on QMGR20). In other words, QMGR20 referecnces a clustered local queue that resides on QMGR10. Anyway, I will probably open another PMR.
Back to top
View user's profile Send private message
tczielke
PostPosted: Thu Oct 23, 2014 5:28 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

Just to close (somewhat) the loop on this thread, after a few days of one server constantly reporting these AMQ9456 errors, they seem to then go away. I have seen other servers display this behavior, too. This does not seem to be a connectivity issue between the PRs and FRs. Weird.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » Clustering » AMQ9456 Question
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.