| Author |
Message
|
| zhanghz |
 Posted: Tue Jun 17, 2008 10:15 pm Post subject: (Closed) CLUSSDR RETRYING after adding a QMGR to CLUSTER |
 |
|
Disciple
Joined: 17 Jun 2008
Posts: 186
|
Hi, I am learning QMGR clustering. I downloaded the IBM book Queue Manager Clusters (SC34-6589-00) and followed the steps in Task 2a to add a new QMGR to my cluster that I built following steps in Task 1.
But the CLUSSDR I defined on the 3rd QMGR is always RETRYING. I can't fingure out why, i believe i followed the steps in the book, but something must be wrong. Please help.
My test is done using Windows WMQ v6. All QMGRs are created in the same machine (my laptop).
I first followed the steps in Task 1 to create a FR with 2 QMGRs. This works fine. (As I already created QMGRs following the help manual that comes with WMQ6, my QMGR names are QM_APPLE (listening at 1414) & QM_ORRANGE (listening at 1415) instead of LONDON & NEWYORK, and my CLUSTER's name is MYCLUSTER instead of INVENTORY.)
I created 2 QCs on QM_APPLE: QM_APPLE.CLUSQ1 & CLUSQ.SAMENAME.1; similarly 2 QCs on QM_ORRANGE: QM_ORRANGE.CLUSQ1 & CLUS1.SAMENAME.1 (both QMGRs have a QC with this same name).
Then I followed the steps in Task 2a to add a 3rd QMGR (QM_BANANA (listening at 1416) intead of PARIS) to the repository. I chose to connect the new QMGR to QM_APPLE. So I defined (on QM_BANANA) CLUSSDR TO.APPLE.CLUSRC with CONNAME = LOCALHOST(1414) and CLUSRCVR TO.BANANA.CLUSRC with CONNAME = LOCALHOST(1416). As all my 3 QMGRs were running already, I didn't do anything else. I waited for a while, checked, the CLUSSDR TO.APPLE.CLUSRC is always RETRYING, the CLUSRCVR is INACTIVE. I restarted all 3 QMGRs too, still the same problem.
Note sure what caused this and how to check. Please guide me.
Below are some messages from error log of QM_BANANA:
| Quote: |
-------------------------------------------------------------------------------
18/06/2008 12:58:27 - Process(4676.1) User(MUSR_MQADMIN) Program(amqzmgr0.exe)
AMQ5026: The Listener 'LISTENER.TCP' has started. ProcessId(2828).
EXPLANATION:
The Listener process has started.
ACTION:
None.
-------------------------------------------------------------------------------
18/06/2008 12:58:31 - Process(5804.1) User(MUSR_MQADMIN) Program(runmqchi.exe)
AMQ8024: WebSphere MQ channel initiator started.
EXPLANATION:
The channel initiator for queue SYSTEM.CHANNEL.INITQ has been started.
ACTION:
None.
-------------------------------------------------------------------------------
18/06/2008 12:58:35 - Process(784.2) User(MUSR_MQADMIN) Program(amqrmppa.exe)
AMQ9002: Channel 'TO.APPLE.CLUSRC' is starting.
EXPLANATION:
Channel 'TO.APPLE.CLUSRC' is starting.
ACTION:
None.
-------------------------------------------------------------------------------
18/06/2008 12:58:40 - Process(784.2) User(MUSR_MQADMIN) Program(amqrmppa.exe)
AMQ9520: Channel not defined remotely.
EXPLANATION:
There is no definition of channel 'TO.APPLE.CLUSRC' at the remote location.
ACTION:
Add an appropriate definition to the remote hosts list of defined channels and
retry the operation.
----- amqrfpta.c : 340 --------------------------------------------------------
18/06/2008 12:58:40 - Process(784.2) User(MUSR_MQADMIN) Program(amqrmppa.exe)
AMQ9999: Channel program ended abnormally.
EXPLANATION:
Channel program 'TO.APPLE.CLUSRC' ended abnormally.
ACTION:
Look at previous error messages for channel program 'TO.APPLE.CLUSRC' in the
error files to determine the cause of the failure.
----- amqrccca.c : 777 -------------------------------------------------------- |
Below are some messages from error log of QM_APPLE:
| Quote: |
-------------------------------------------------------------------------------
18/06/2008 12:58:24 - Process(4868.1) User(MUSR_MQADMIN) Program(amqzmgr0.exe)
AMQ5026: The Listener 'LISTENER.TCP' has started. ProcessId(5144).
EXPLANATION:
The Listener process has started.
ACTION:
None.
-------------------------------------------------------------------------------
18/06/2008 12:58:29 - Process(2532.1) User(MUSR_MQADMIN) Program(runmqchi.exe)
AMQ8024: WebSphere MQ channel initiator started.
EXPLANATION:
The channel initiator for queue SYSTEM.CHANNEL.INITQ has been started.
ACTION:
None.
-------------------------------------------------------------------------------
18/06/2008 12:58:40 - Process(5168.2) User(MUSR_MQADMIN) Program(amqrmppa.exe)
AMQ9519: Channel 'TO.APPLE.CLUSRC' not found.
EXPLANATION:
The requested operation failed because the program could not find a definition
of channel 'TO.APPLE.CLUSRC'.
ACTION:
Check that the name is specified correctly and the channel definition is
available.
----- amqrcoba.c : 1187 -------------------------------------------------------
18/06/2008 12:58:40 - Process(5168.2) User(MUSR_MQADMIN) Program(amqrmppa.exe)
AMQ9999: Channel program ended abnormally.
EXPLANATION:
Channel program 'TO.APPLE.CLUSRC' ended abnormally.
ACTION:
Look at previous error messages for channel program 'TO.APPLE.CLUSRC' in the
error files to determine the cause of the failure.
----- amqrmrsa.c : 468 -------------------------------------------------------- |
Last edited by zhanghz on Fri Jun 20, 2008 4:09 am; edited 3 times in total |
|
| Back to top |
|
 |
| zhanghz |
| Posted: Tue Jun 17, 2008 10:22 pm Post subject: |
|
|
Disciple
Joined: 17 Jun 2008
Posts: 186
|
just a footnote, this is my 2nd attempt already. 1st attempt same problem (several days ago), so i deleted the QMGR (same name QM_BANANA) and created this new one yesterday. I just noticed there is some procedure for deleting a QMGR from a cluster, I guess i didn't follow it. I guess that's why i can now see my old CLUSTER CHANNELS defined in my 1st attemp in some entries in the error logs.
I hope this does not complicate the problem. Thanks. |
|
| Back to top |
|
|
| exerk |
| Posted: Tue Jun 17, 2008 11:39 pm Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
I may be stating the obvious here as you may already have checked this, if so my apologies...
The CLUSSDR on QM_BANANA must match the name of the existing CLUSRCVR on QM_APPLE
I suspect this is not the case as your error log is reporting:
| Quote: |
18/06/2008 12:58:40 - Process(784.2) User(MUSR_MQADMIN) Program(amqrmppa.exe)
AMQ9520: Channel not defined remotely.
EXPLANATION:
There is no definition of channel 'TO.APPLE.CLUSRC' at the remote location.
ACTION:
Add an appropriate definition to the remote hosts list of defined channels and
retry the operation.
|
There may be a typographical error in the channel name.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| zhanghz |
| Posted: Wed Jun 18, 2008 12:32 am Post subject: |
|
|
Disciple
Joined: 17 Jun 2008
Posts: 186
|
Hi exerk, but for CLUSTERs I will only need to define CLUSSDR and CLUSRCVR on the new QMGR that I am adding to the existing cluster. The corresponding CLUSRCVR and CLUSSDR are supposed to be created automatically on the QMGR that is within the FR. If we need to define the CLUSSDR & CLUSRCVR on the existing QMGR manually, that will defeat the purpose of the whole cluster concept.
But, i did try to create a CLUSRCVR channel in QM_APPLE manually in my first attempt. But that didn't solve the problem, i got the following error in QM_APPLE error log:
| Quote: |
17/06/2008 13:33:38 - Process(5344.86) User(MUSR_MQADMIN) Program(amqrmppa.exe)
AMQ9002: Channel 'BANANA.APPLE.CLUSRC' is starting.
EXPLANATION:
Channel 'BANANA.APPLE.CLUSRC' is starting.
ACTION:
None.
-------------------------------------------------------------------------------
17/06/2008 13:33:38 - Process(5344.86) User(MUSR_MQADMIN) Program(amqrmppa.exe)
AMQ9587: Program cannot open queue manager object.
EXPLANATION:
The attempt to open object '%CHLBATCH.18' on queue manager 'QM_APPLE' failed
with reason code 2101.
ACTION:
Use the standard facilities supplied with your system to record the problem
identifier. Contact your IBM support center.
----- amqrsyna.c : 1070 -------------------------------------------------------
17/06/2008 13:33:38 - Process(5344.86) User(MUSR_MQADMIN) Program(amqrmppa.exe)
AMQ9999: Channel program ended abnormally.
EXPLANATION:
Channel program 'BANANA.APPLE.CLUSRC' ended abnormally.
ACTION:
Look at previous error messages for channel program 'BANANA.APPLE.CLUSRC' in
the error files to determine the cause of the failure.
----- amqrmrsa.c : 468 -------------------------------------------------------- |
and correspondingly some error in QM_BANANA. as i deleted the QM_BANANA i created in my 1st attempt, i don't have the exact error for them now.
I tried to rename the SYNC file and restarted QM_BANANA, but CLUSSDR on QM_BANANA still retrying. That's when I deleted QM_BANANA and re-created it for my 2nd attempt on this cluster thing.
But, same problem, as stated in my post above. |
|
| Back to top |
|
|
| exerk |
| Posted: Wed Jun 18, 2008 12:48 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
I think you may have misunderstood my post, my apologies if I was not very clear.
It may be easier for you to scrap the current set-up and begin from scratch...
1. When you created your first two queue managers (the repositories) you will have defined a CLUSSDR and CLUSRCVR from each, to the other.
2. When you create the third queue manager you must create a CLUSRCVR for that queue manager, and a CLUSSDR to one of the repositories - that CLUSSDR must be the same name as the CLUSRCVR of the repository you are joining the third queue manager to.
So:
FULL REPOSITORY
Qmgr A CLUSRCVR = TO.QMGRA
Qmgr A CLUSSDR = TO.QMGRB
FULL REPOSITORY
Qmgr B CLUSRCVR = TO.QMGRB
Qmgr B CLUSSDR = TO.QMGRA
PARTIAL REPOSITORY
Qmgr C CLUSRCVR = TO.QMGRC
Qmgr C CLUSSDR = TO.QMGRA
As a general rule, I normally define a CLUSRCVR before a CLUSSDR.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| zhanghz |
| Posted: Wed Jun 18, 2008 1:18 am Post subject: |
|
|
Disciple
Joined: 17 Jun 2008
Posts: 186
|
i see , i see, it's re-using the CLUSRCVR on QM_APPLE !!! This is where I missed it...
I changed, now need to add SSL to the CLUSSDR on QM_BANANA. Should have no more problem.. Thanks a lot! Cleared my doubt. |
|
| Back to top |
|
|
| zhanghz |
| Posted: Wed Jun 18, 2008 1:21 am Post subject: |
|
|
Disciple
Joined: 17 Jun 2008
Posts: 186
|
hahaha, working now.. Finally, thanks exerk!! thank you for pointing out the obvious!! Self-learning is not easy, haha..
cheers  |
|
| Back to top |
|
|
| exerk |
| Posted: Wed Jun 18, 2008 1:43 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
I'm glad to have helped - sometimes a 'fresh' pair of eyes helps 
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| zhanghz |
| Posted: Thu Jun 19, 2008 9:00 pm Post subject: |
|
|
Disciple
Joined: 17 Jun 2008
Posts: 186
|
Hi guys, new question now.
Remember I have done 2 attempts on clustering? Ok, the problem is, the old CLUSRCVR APPLE.BANANA.CLUSRC that I set up on QM_BANANA in my 1st attemp is still inside QM_APPLE's repository (ya, i didn't follow the steps when I deleted the QM_BANANA in my 1st attempt). I can't get rid of it now even using RESET CLUSTER command.
Here is what I tryied to remove the old QM_BANANA channel from QM_APPLE's repository:
issue RESET CLUSTER(MYCLUSTER) ACTION(FORCEREMOVE) QMID('QM_BANANA_2008-06-11_14.02.59') QUEUEUS(YES) (I tried with and without QUEUES(YES))
I got this:
| Quote: |
----- amqrcmna.c : 915 --------------------------------------------------------
19/06/2008 21:45:34 - Process(4508.1) User(MUSR_MQADMIN) Program(amqrrmfa.exe)
AMQ9405: FORCEREMOVE QUEUES(YES) command processed, cluster MYCLUSTER target
QM_BANANA_2008-06-11_14.02.59.
EXPLANATION:
The repository queue manager successfully processed a RESET ACTION(FORCEREMOVE)
command with the QUEUES(YES) option for the indicated cluster and target queue
manager.
ACTION:
None.
-------------------------------------------------------------------------------
19/06/2008 21:45:36 - Process(4456.2) User(MUSR_MQADMIN) Program(amqrmppa.exe)
AMQ9002: Channel 'APPLE.ORRANGE.CLUSRC' is starting.
EXPLANATION:
Channel 'APPLE.ORRANGE.CLUSRC' is starting.
ACTION:
None.
-------------------------------------------------------------------------------
19/06/2008 21:54:11 - Process(4508.1) User(MUSR_MQADMIN) Program(amqrrmfa.exe)
AMQ9455: FORCEREMOVE command failed, cluster MYCLUSTER, target
QM_BANANA_2008-06-11_14.02.59, not found.
EXPLANATION:
The repository queue manager could not process a RESET ACTION(FORCEREMOVE)
command for the indicated cluster and target queue manager, because no
information about that queue manager was found in the local repository. The
command is ignored.
ACTION:
Reissue the command, specifying the correct queue manager name or identifier.
-------------------------------------------------------------------------------
......
19/06/2008 22:09:42 - Process(4508.1) User(MUSR_MQADMIN) Program(amqrrmfa.exe)
AMQ9455: FORCEREMOVE command failed, cluster MYCLUSTER, target
QM_BANANA_2008-06-11_14.02.59, not found.
EXPLANATION:
The repository queue manager could not process a RESET ACTION(FORCEREMOVE)
command for the indicated cluster and target queue manager, because no
information about that queue manager was found in the local repository. The
command is ignored.
ACTION:
Reissue the command, specifying the correct queue manager name or identifier.
-------------------------------------------------------------------------------
19/06/2008 22:10:26 - Process(4508.1) User(MUSR_MQADMIN) Program(amqrrmfa.exe)
AMQ9507: Channel 'APPLE.BANANA.CLUSRC' is currently in-doubt.
EXPLANATION:
The requested operation cannot complete because the channel is in-doubt with
host 'QM_BANANA'.
ACTION:
Examine the status of the channel, and either restart a channel to resolve the
in-doubt state, or use the RESOLVE CHANNEL command to correct the problem
manually.
----- amqrcmna.c : 915 -------------------------------------------------------- |
For "target QM_BANANA_2008-06-11_14.02.59, not found.", I thought the QM_BANANA was already cleared/removed, but DIS CLUSQMGR(*) still showed the old QM_BANANA 's info:
| Quote: |
DIS CLUSQMGR(*)
1 : DIS CLUSQMGR(*)
AMQ8441: Display Cluster Queue Manager details.
CLUSQMGR(QM_APPLE) CHANNEL(ORRANGE.APPLE.CLUSRC)
CLUSTER(MYCLUSTER)
AMQ8441: Display Cluster Queue Manager details.
CLUSQMGR(QM_BANANA) CHANNEL(TO.BANANA.CLUSRC)
CLUSTER(MYCLUSTER)
AMQ8441: Display Cluster Queue Manager details.
CLUSQMGR(QM_BANANA) CHANNEL(APPLE.BANANA.CLUSRC)
CLUSTER(MYCLUSTER)
AMQ8441: Display Cluster Queue Manager details.
CLUSQMGR(QM_ORRANGE) CHANNEL(APPLE.ORRANGE.CLUSRC)
CLUSTER(MYCLUSTER) |
I tried to DIS CHL(APPLE.BANANA.CLUSRC), it says "AMQ8147: WebSphere MQ object APPLE.BANANA.CLUSRC not found.". But a RESOLVE CHANNEL(APPLE.BANANA.CLUSRC) ACTION(COMMIT) ran ok. But DIS CLUSQMGR(*) will still show the old QM_BANANA.
I also did REFRESH CLUSTER (REPOS(NO) as a FR qmgr, and repos(yes) as a PR qmgr), no use.
What can I do now to remove the out-dated information? Pls advise. Thanks. |
|
| Back to top |
|
|
| Nigelg |
| Posted: Thu Jun 19, 2008 9:39 pm Post subject: |
|
|
Grand Master
Joined: 02 Aug 2004
Posts: 1046
|
You have made a right old mess of it all.
Delete all the qmgrs and start again.
HINT: Think first, then act.
_________________
MQSeries.net helps those who help themselves.. |
|
| Back to top |
|
|
| zhanghz |
| Posted: Fri Jun 20, 2008 12:43 am Post subject: |
|
|
Disciple
Joined: 17 Jun 2008
Posts: 186
|
Yes, i think i can make it right if i do it again from scratch. But what if this is production environment?
That's the rationale of me asking for advise here actually. i want to know whether and how i can do this without deleting all qmgrs. I believe for a matured product, there must be a way of doing everything.
and yes, i know the repository will remove the out-dated info after certain days. that's not what i am after here. and i don't want to wait for that long.
Thanks. |
|
| Back to top |
|
|
| exerk |
| Posted: Fri Jun 20, 2008 1:02 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
I would hope that a Production environment would never have got this far up the creek without a paddle 
I concur with Nigelg - scrap it and start again.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| zhanghz |
| Posted: Fri Jun 20, 2008 2:30 am Post subject: |
|
|
Disciple
Joined: 17 Jun 2008
Posts: 186
|
hahaha, one never knows what will happen..
but anyway, i will start from scratch.
thanks all. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Jun 20, 2008 1:09 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| But what if this is production environment? |
Do your development of new qmgrs and clusters in a test environment. Create object definitions with MQSC scripts. Once you get a good cluster, migrate the script to QA, and then to production.
If you need to alter or modify the cluster, test these changes in your test environment - again with MQSC scripts; then migrate the changes to production.
Read the WMQ Clusters manual. Follow the step-by-step instructions in the manual.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
|
|
