| Author |
Message
|
| tonythomasantony |
 Posted: Mon Aug 04, 2014 8:43 pm Post subject: Multi Instance Broker: Failover Testing |
 |
|
Apprentice
Joined: 12 Aug 2006
Posts: 47
|
Hi ,
I am in a process to set up multi instance setup in Windows 2008 R2 OS.
I have followed exactly the prcoedure mentioned for multi instance queue manager and broker, mentioned in IIB/MQv7.5 documentation.
But during fail over testing of multi instance broker, by POWEROFF one of the server, the broker created in the live server shows "stopped" status but messages are prcoessing. And also i can see 2 bipbroker,2 bipservice and 2 biphttplistener prcesses in the task manager. No error reported in evenvwr and mq and broker logs.
Please let me know if i need to provide more details for the problem statement if the below informations are not sufficient.
Env details
$$$$$$$$$$
Windows 2008 R2
IIB and MQ 7.5
Queue managers are in cluster.
Net result after POWEROFF of one of the server is
a. Queue managers failed over sucessfully and running fine.
b. Status of Broker which is created in non-Poweroff server shows stopped status in mqsilist.
c. When i try to start the broker i have received an error saying that the start command cannot applied to a running broker.
d. I can able to stop and then start successfully the same broker.
Before POWEROFF
**************
Server1 have one queue manager created with name SVR1_QM and instance of queue manager created in server2,SVR2_QM
Server2 have one queue manager created with name SVR2_QM and instance of queue manager created in server1,SVR1_QM
Server1 have one broker created with name SVR1_IIB and instance of broker created in server2,SVR2_IIB
Server2 have one broker created with name SVR2_IIB and instance of broker created in server1,SVR1_IIB
Server1 Queuemanager Status
--------------------------
SVR1_QM Running
SVR2_QM (Standby)
Server2 Queuemanager Status
--------------------------
SVR2_QM Running
SVR1_QM (Standby)
Server1 Broker Status
--------------------------
SVR1_IIB Running
SVR2_IIB (Standby)
Server2 Broker Status
--------------------------
SVR2_IIB Running
SVR1_IIB (Standby)
Applied POWEROFF on Server1
After POWEROFF
**************
Server2 Queuemanager Status
--------------------------
SVR2_QM Running
SVR1_QM Running
Server2 Broker Status
--------------------------
SVR2_IIB Stopped
SVR1_IIB Running
Regards,
Tony Thomas |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Mon Aug 04, 2014 10:26 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
without knowing exactly how you have configured the MSCS cluster resources it is difficult to point you in the right direction.
If I were doing this then the two pairs of broker & qmgrs would configured in TWO different cluster services.
Then I would be able to test the living daylights out of each one independently.
Start with just the queue managers. As the brokers will be dependent upon them you have to make 1000% sure that the cluster resources (disk, ip and QMGR) are failing correctly.
Then and only then add the brokers into the equation. Make sure that one pair works perfectly before adding the second.
Make sure that the dependencies are correctly defined AND that the correct params are used on the broker creation so that it works properly in the domain. This is especially true if the DC is running sever 2012. There are some new options added to the mqsicreatebroker command (-B )
Manually fail over the separate clusters. When that is working and then and only then start node shutdown/powerfailure testing.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Aug 04, 2014 10:57 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
| smdavies99 wrote: |
without knowing exactly how you have configured the MSCS cluster resources it is difficult to point you in the right direction.
If I were doing this then the two pairs of broker & qmgrs would configured in TWO different cluster services.
Then I would be able to test the living daylights out of each one independently.
Start with just the queue managers. As the brokers will be dependent upon them you have to make 1000% sure that the cluster resources (disk, ip and QMGR) are failing correctly.
Then and only then add the brokers into the equation. Make sure that one pair works perfectly before adding the second.
Make sure that the dependencies are correctly defined AND that the correct params are used on the broker creation so that it works properly in the domain. This is especially true if the DC is running sever 2012. There are some new options added to the mqsicreatebroker command (-B )
Manually fail over the separate clusters. When that is working and then and only then start node shutdown/powerfailure testing. |
I don't know where you got the impression that he was running MSCS.
I was under the impression he was running multi-instance qmgrs and multi-instance brokers.
To be fully safe and sure you would be running the brokers as qmgr services, starting when the qmgr starts and stopping when the qmgr stops...
Running the brokers as multi-instance running / standby just as the qmgr is possible but you better test the hell out of that configuration.
And remember to make sure you use the extra parameter (-B) at creation of the broker / add broker instance...
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Mon Aug 04, 2014 11:12 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| fjb_saper wrote: |
I don't know where you got the impression that he was running MSCS.
I was under the impression he was running multi-instance qmgrs and multi-instance brokers.
|
Clearly not had enough  yet after a late night sorting a production problem half a world away.
The other thing to note is that the IBM docs are IMHO sorely lacking in this area. There are holes that you will find out through experimentation. As such it is essential that you capture everything you do as you go along so that you can document it and then tear everything down and start again following only the instructions in the document. Trying to cut corners here will only come back to bite you badly later on.
As my colleague says, you have to test the hell out of this.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| tonythomasantony |
| Posted: Tue Aug 05, 2014 12:36 am Post subject: |
|
|
Apprentice
Joined: 12 Aug 2006
Posts: 47
|
Hi smdavies99,fjb_saper,
Thanks a lot for the reply. Let me point out some more details regarding the situation.
1.Servers are not in MSCS.
2.We have one dedicated file server and one domain controller server.
3. Dedicated file server hosts the shared workpath for queue managers and brokers.
4. The broker is created with -B option.
5. Failover of BOTH of the queue managers working fine with endmqm - S option.
6. Failover of BOTH of the queue managers working fine with POWER OFF the servers.
7. Failover of BOTH of the brokers working fine with endmqm - S option.
8. a. Failover of Broker which was running in the POWEROFF server is running in the live server and showing the correct status.
8.b.Broker which was running in the non-POWEROFF server changed the status to STOPPED once the other went down, But runnign all the process and processing messages without any issue.
9.NOT running the brokers as qmgr services.
Regards,
Tony Thomas |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Aug 05, 2014 1:39 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| tonythomasantony wrote: |
9.NOT running the brokers as qmgr services.
|
and
| fjb_saper wrote: |
To be fully safe and sure you would be running the brokers as qmgr services, starting when the qmgr starts and stopping when the qmgr stops...
|
Perhaps that might be worth investigating especially as the broker (at least up to and inclusing IIB9) needs a queue manager to operate. Making the broker a virtual child of the queue manager seems to me to be the way to go.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
|
|
