| Author |
Message
|
| Deas |
 Posted: Sat Sep 24, 2011 12:42 pm Post subject: 2008 r2 cluster with mq 7 - resource start/stop not working |
 |
|
Novice
Joined: 24 Sep 2011
Posts: 11
|
hello,
we migrated a 2003 mscs with mq 6 and 5 queue manager to a new 2008 r2 mscs with mq 7.0.1.6. everything worked fine so far and it was no problem failing over the resources from one node to the other.
a day later i saw that 2 of the 5 queue managers are failed in cluster manager and i was unable to start them in cluster manager. also after a restart of the whole cluster this was not working. but for all 5 queue managers!
now i have the following situation: with cluster manager i am not able to start/stop the queue managers. with mq explorer i can start/stop the queue managers without any problem.
has anybody an idea what is causing this problem?
brgds Deas |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Sat Sep 24, 2011 1:46 pm Post subject: Re: 2008 r2 cluster with mq 7 - resource start/stop not work |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Deas wrote: |
... i was unable to start them in cluster manager. ...
now i have the following situation: with cluster manager i am not able to start/stop the queue managers. with mq explorer i can start/stop the queue managers without any problem.
has anybody an idea what is causing this problem?
brgds Deas |
You could begin by providing some detail as to why you were unable to start the qmgrs. What error indications did you receive when you issued the strmqm command, for example? What did the error logs tell you?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Deas |
| Posted: Sat Sep 24, 2011 2:43 pm Post subject: |
|
|
Novice
Joined: 24 Sep 2011
Posts: 11
|
you quote the details and ask for them??
i don´t know why i am unable to start the queue managers via MS CLUSTER MANAGER. if i´d know the reason i could fix it. the cluster manager just times out and nothing happens when i look into mq explorer while the cluster tries to start them. also the cluster.log just logs this:
| Quote: |
| RhsCall::DeadlockMonitor: Call ONLINERESOURCE timed out for resource 'IBM MQ xxxxx'. |
when the cluster gave up after a few minutes i can start the queue managers with MQ EXPLORER. there is NO error.
so the problem looks like as if the cluster is not able to start/stop the queue managers.
brgds Deas |
|
| Back to top |
|
|
| exerk |
| Posted: Sat Sep 24, 2011 3:03 pm Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
Have your System Admins take a look as I suspect the MSCS cluster may not be set up correctly. Also, just how exactly did you 'migrate'?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| Deas |
| Posted: Sat Sep 24, 2011 3:21 pm Post subject: |
|
|
Novice
Joined: 24 Sep 2011
Posts: 11
|
i am the system admin - the cluster is set up correct. this is not the first 2008 r2 cluster we run.
the migration was done with an engineer from ibm. we exported the config of the queue managers with the latest version of saveqmgr and imported it on the new cluster with runmqsc.
as i said - it was running fine for about a day. then i saw that 2 of the queue managers were failed in the cluster and from that point the management via the cluster manager was no longer possible. i also tried to delete one of the cluster resources and recreated it. did not changed the behaviour. the engineer from ibm also took a look at the mq logfiles and found nothing that indicates a problem. unfortunately he is no mq cluster specialist so he tries to ask some of his colleagues if they ever had such an issue. if nothing helps we must open a support call with ibm next week.
btw. - we did the setup exactly as described here:
http://www-01.ibm.com/support/docview.wss?uid=swg21251099
http://publib.boulder.ibm.com/infocenter/wmqv7/v7r0/index.jsp?topic=%2Fcom.ibm.mq.amqzag.doc%2Ffa14240_.htm&resultof=%22test%22%20%22failover%22%20%22failov%22
brgds Deas |
|
| Back to top |
|
|
| exerk |
| Posted: Sun Sep 25, 2011 3:50 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
Are the queue managers set to start manually and not automatically?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| Deas |
| Posted: Sun Sep 25, 2011 10:24 am Post subject: |
|
|
Novice
Joined: 24 Sep 2011
Posts: 11
|
yes, startup of the queue managers is service (manual).
brgds Deas |
|
| Back to top |
|
|
| exerk |
| Posted: Sun Sep 25, 2011 10:29 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
Personally, I'd take the queue managers out of cluster control, take down the cluster and start again. This has the 'smell' of a hardware/OS issue to me (but always happy to be proved wrong).
Alternately, as you have stated this is not the first migration you've done, but possibly the first not-completely-successful one, cross-compare (minutely) the problem setup with a working one.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| Deas |
| Posted: Sun Sep 25, 2011 11:04 am Post subject: |
|
|
Novice
Joined: 24 Sep 2011
Posts: 11
|
unfortunately option 1 is not possible. this ibm mq server is part of a resource where msmq and a application is also part of the cluster group.
what i also saw today is that the application had a crash due to invalid parameters in the application database and so the cluster did a failover to the other node. i was quite surprised as i saw all 5 queue managers online. after we resolved the database issue i did a failover to the second node to see if it is working no, but all 5 queue managers did not come online. also a fallback to the node where it was working 2 minutes before showed the same. all 5 did not come online with the cluster.
very strange...
let´s see what ibm support is telling me this week...
brgds Deas |
|
| Back to top |
|
|
| exerk |
| Posted: Sun Sep 25, 2011 1:39 pm Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
Are any of the queue managers tied to XA resources, e.g. the database? Is MSMQ working fine? Is Biztalk in the mix somewhere?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| Deas |
| Posted: Sun Sep 25, 2011 10:37 pm Post subject: |
|
|
Novice
Joined: 24 Sep 2011
Posts: 11
|
no - the dependencies are the disk and the network name (and so also the ip address). biztalk is not installed on this system - but yes, we have it! the database of the application is not local on this cluster, it is on a redhat cluster with oracle 11g.
msmq is working fine and the failover is working without any problem.
the application that was easy to migrate (ibm mq) makes now problems, the application that was a nightmare to migrate (msmq) runs fine.
if you ever must migrate msmq from 2003 to 2008 make sure to note not only the config and the permissions you set, also include the default permissions. the default permissions where changed and some of our applications access with a named ad user, but i had to set anonymous or default user with send permissions to get them working. and there is no migration tool from microsoft. i opened a advisory call with microsoft for that because i found nothing that could do the job. and there is nothing. you could only develop a c# application to do it. so i noted from 41 queues the config by hand and then created 41 queues on the new server by hand. and due to the changed default permissions and maybe the programming knowledge of the app developers i had to change nearly all queues 2-3 times... 
brgds Deas |
|
| Back to top |
|
|
| exerk |
| Posted: Mon Sep 26, 2011 12:32 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
So, let me get this straight:
1. You backed up the queue manager objects and authorities using the MS03 SupportPac?
2. You noted all queue manager-related Windows Registry settings and other associated settings?
3. You recreated the W2K3 setup on W2K8 exactly, allowing for the subtle differences between the two platforms?
You have now also mentioned MSMQ, so is the MSMQ-MQ Bridge being used?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| Deas |
| Posted: Mon Oct 03, 2011 1:17 am Post subject: |
|
|
Novice
Joined: 24 Sep 2011
Posts: 11
|
i can´t say exactly what caused the problem, but we created one of the queue managers new last week and it is working now.
the guy from ibm mq support said that it has probably to do with how i initially created the queue managers. i made a small batch job where i entered all the commands i´d do by hand. but he said that when i use strmqm to do the first start the queue manager is started in the context of my user account and that is not a good idea. this is something special for windows he said.
so i should start the queue manager with mq explorer or with the option -ss.
i did that for one queue manager last week (deleted and recreated it) and this one is working fine so far. but he said that i did nothing wrong when creating the queue managers. this is the only thing he could think of that caused the prolem.
brgds Deas |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Oct 06, 2011 3:55 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| Deas wrote: |
so i should start the queue manager with mq explorer or with the option -ss.
|
No.
http://publib.boulder.ibm.com/infocenter/wmqv7/v7r0/topic/com.ibm.mq.amqzag.doc/fa14400_.htm
Review this whole section in the Info Center on MQ and MSCS. Maybe you'll spot something wrong.
Is it only the QM resource that does not come online, or do other resources also fail to come online?
Are all 5 of these QMs in the same MSCS cluster group? Or do you have separate groups with each having their own QM, disk, IP, etc?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Oct 06, 2011 7:35 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
I've had strange results with starting qmgrs on Windows at V7, even using amqmdain. I found that they work best if started with
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
