| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| QM fail over..... |
« View previous topic :: View next topic » |
| Author |
Message
|
| pinnacle |
 Posted: Wed Jul 08, 2009 6:05 am Post subject: QM fail over..... |
 |
|
Newbie
Joined: 07 Jul 2009
Posts: 1
|
| If two Queue Managers are in Cluster and they are sharing load.If any of the QM goes down while processing say 1000 messages will those 1000 messages be shifted (fail over) to the other running QM or those will be available only after the QM goes up ? |
|
| Back to top |
|
 |
| exerk |
| Posted: Wed Jul 08, 2009 6:19 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
NO!. Clustering provides load-balancing, it does not, and cannot provide fail-over. What do expect a failed queue manager to do? Somehow lever off the disk platter cover and chisel off the electrons from the substrate? And somehow ship them over to the other queue manager? The messages will remain unavailable for processing until the failed queue manager is restarted - CAVEAT: provided the messages are set to be PERSISTENT.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| aditya.aggarwal |
| Posted: Wed Jul 08, 2009 6:59 am Post subject: |
|
|
Master
Joined: 13 Jan 2009
Posts: 252
|
| Quote: |
| Clustering provides load-balancing, it does not, and cannot provide fail-over. |
If we cloned the queue managers and put them in to same MQ cluster then it will provide failover..
i.e. qmgr A, qmgr B are in cluster and need failover.... Create a cloned qmgr C of qmgr A and cloned qmgr D of qmgr B
put qmgr A, B, C and D in to same Cluster.. Tune your application in such a way that it can process the message for qmgr A and C sametime...
and qmgr B and D at same time... this can be acheived by load balancer ...
now if qmgr A/server goes down ...qmgr C will be active and process the messages...
same with qmgr B and D compbination...
Cheers,
Aditya |
|
| Back to top |
|
|
| exerk |
| Posted: Wed Jul 08, 2009 7:07 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| aditya.aggarwal wrote: |
| Quote: |
| Clustering provides load-balancing, it does not, and cannot provide fail-over. |
If we cloned the queue managers and put them in to same MQ cluster then it will provide failover..
i.e. qmgr A, qmgr B are in cluster and need failover.... Create a cloned qmgr C of qmgr A and cloned qmgr D of qmgr B
put qmgr A, B, C and D in to same Cluster.. Tune your application in such a way that it can process the message for qmgr A and C sametime...
and qmgr B and D at same time... this can be acheived by load balancer ...
now if qmgr A/server goes down ...qmgr C will be active and process the messages...
same with qmgr B and D compbination...
Cheers,
Aditya |
Sage advice...and TOTALLY WRONG! What, exactly, do you mean by:
| Quote: |
| If we cloned the queue managers and put them in to same MQ cluster then it will provide failover.. |
because it equates with nothing that I understand of clustering. If, however, you mean evenly distributing copies of the same message (pub/sub anyone?) that may work - provided of course you have some method of dealing with the countless duplicates, and are happy to have that solution flooding your network.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 08, 2009 7:15 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| aditya.aggarwal wrote: |
| Quote: |
| Clustering provides load-balancing, it does not, and cannot provide fail-over. |
If we cloned the queue managers and put them in to same MQ cluster then it will provide failover.. |
No it will not! 
| aditya.aggarwal wrote: |
| i.e. qmgr A, qmgr B are in cluster and need failover.... Create a cloned qmgr C of qmgr A and cloned qmgr D of qmgr B |
I will assume you mean "create a queue manager configured the same way". If you "clone" a queue manager (i.e. copy it) and put it in the same cluster the cluster will break.
| aditya.aggarwal wrote: |
put qmgr A, B, C and D in to same Cluster.. Tune your application in such a way that it can process the message for qmgr A and C sametime...
and qmgr B and D at same time... this can be acheived by load balancer ... |
No it can't! No message once sent to A will ever be delivered to C no matter what you do with the load balancing. You also can't connect an application to A & C sametime; you can use a client table and some application code to reconnect to C if A fails but it won't get the same message that A has.
| aditya.aggarwal wrote: |
| now if qmgr A/server goes down ...qmgr C will be active and process the messages... |
I repeat, C will not get any messages that have already been delivered (or queued to be delivered) to A. This is particually relevant here as the original poster specifically asked if 1000 messages held on the failed queue manager would be failed over. Your post suggests that they would move from A to C because of the load balancing. This is a very misleading statement.
This topic has been widely discussed on the forum.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Jul 08, 2009 7:44 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Vitor wrote: |
| No message once sent to A will ever be delivered to C no matter what you do with the load balancing. |
I'm going to quibble here.
IF a message has been SENT to A, but not DELIVERED to A (i.e. still on the S.C.T.Q) AND the message has been sent with BIND_NOT_FIXED, AND the channel to A goes away, the cluster CAN attempt to ship the message to C instead of A.
But Once the message has been DELIVERED to A, nothing will ever move it to C. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 08, 2009 7:53 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqjeff wrote: |
| Vitor wrote: |
| No message once sent to A will ever be delivered to C no matter what you do with the load balancing. |
I'm going to quibble here. |
This is a serious quibble.
| mqjeff wrote: |
| IF a message has been SENT to A, but not DELIVERED to A (i.e. still on the S.C.T.Q) AND the message has been sent with BIND_NOT_FIXED, AND the channel to A goes away, the cluster CAN attempt to ship the message to C instead of A. |
That's a fair number of pre-conditions, and has a few variables in it, but I will accept the rather hair-splitting point you're making.
| mqjeff wrote: |
| But Once the message has been DELIVERED to A, nothing will ever move it to C. |
Which was the original question here, making the weird load balancing advice more dangerous than normal.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| JosephGramig |
| Posted: Wed Jul 08, 2009 9:58 am Post subject: |
|
|
Grand Master
Joined: 09 Feb 2006
Posts: 1244
Location: Gold Coast of Florida, USA
|
Once the message has been delivered, only Hardware High Availability can recover the QMGR, if the equipment that the QMGR is on cannot be restarted.
Note, this kind of HA is not instantaneous recovery and takes time (minutes).
This is the recommended way to go about processing the trapped messages.
Of course, if you are on z/OS, look into queue sharing groups. |
|
| Back to top |
|
|
| exerk |
| Posted: Wed Jul 08, 2009 12:19 pm Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| mqjeff wrote: |
...I'm going to quibble here.
IF a message has been SENT to A, but not DELIVERED to A (i.e. still on the S.C.T.Q) AND the message has been sent with BIND_NOT_FIXED, AND the channel to A goes away, the cluster CAN attempt to ship the message to C instead of A.
But Once the message has been DELIVERED to A, nothing will ever move it to C... |
Fair point, and well made, but I think the 'advice' (please note the quotes) that aditya.aggarwal made is that it is possible to:
1. Clone a queue manager, by which I infer he means replicate the queue objects - accepted.
2. Have an application that can'...can process the message for qmgr A and C sametime...', which I read to mean that the same, i.e. duplicate message, has been received at each queue manager.
3. That this can be achieved by '...load balancer...', which if so, I'd like to see that load balancer (in which I would happily buy shares, as it is obviously the killer app that will finally allow non-WMQ propagation of the same message), or aditya.aggarwal's understanding of queue manager clustering, and my understanding of it, wildly differ.
Quibble aside, I'm with Vitor that the advice given was at best erroneous, bordering on confusing, and at worst, likely to induce apoplexy in the poor individual trying to implement it - though undoubtedly well intentioned.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| aditya.aggarwal |
| Posted: Wed Jul 08, 2009 11:47 pm Post subject: |
|
|
Master
Joined: 13 Jan 2009
Posts: 252
|
exerk wrote:
| Quote: |
Fair point, and well made, but I think the 'advice' (please note the quotes) that aditya.aggarwal made is that it is possible to:
1. Clone a queue manager, by which I infer he means replicate the queue objects - accepted.
2. Have an application that can'...can process the message for qmgr A and C sametime...', which I read to mean that the same, i.e. duplicate message, has been received at each queue manager.
3. That this can be achieved by '...load balancer...', which if so, I'd like to see that load balancer (in which I would happily buy shares, as it is obviously the killer app that will finally allow non-WMQ propagation of the same message), or aditya.aggarwal's understanding of queue manager clustering, and my understanding of it, wildly differ. |
1. True.
2. False...we need not to duplicate the messages here...
3. No it’s not correct…
Now I have elaborated this design below..
Let’s say we have qmgr A and qmgr C clone of each other and both have a Cluster queue called Q1.REQUEST ... this queue is receiving request messages from Z/OS based broker message flow. z/os broker queue manager name is X and qmgr X, A and C are part of same MQ cluster.. X is FR and A and C are PR queue managers... There is one more FR qm called Y present in same MQ Cluster...
Remote Application send Request --> Z/Os Broker [X]--> Broker message flow put this message in to Cluster queue Q1.REQUEST at Solaris queue manager A or C based upon MQ Cluster algorithm--> there is Weblogic application server installed at queue manager A and C server for same application--> the application instance at queue manager A and C using different connection parameter to connect with queue manager A and C-->these two application instance pick the different request message from Q1.REQUEST at A and C and process it to a Web application using the Load Balencer..---> Web Application generate a reply for these requests --> write this reply in to Cluster Queue Q1.REPLY at Broker queue manager X--> Broker message flow process it and send this reply back to Remote Application
Now What happen when queue manager A Goes Down:
Remote Application send Request --> Z/Os Broker [X]--> Broker message flow put this message in to Cluster queue Q1.REQUEST at Solaris queue manager A or C --> Queue manager A Goes Down---> There may be some request pending in to ‘Q1.REQUEST’ at qmgr A and will not process till the qmgr A will come up-> Application instance at qmgr C will keep processing the request messages from the queue Q1.REQUEST --> Pending messages at qmgr A will be processed by application after resolving the problem at qmgr A server.
I think this desing is fulfilling the HA and Failover requirement for request -reply messages.
Cheers,
Aditya |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 08, 2009 11:58 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| aditya.aggarwal wrote: |
| There may be some request pending in to ‘Q1.REQUEST’ at qmgr A and will not process till the qmgr A will come up |
Nice to see you agree with us.
| aditya.aggarwal wrote: |
| I think this desing is fulfilling the HA and Failover requirement for request -reply messages. |
You might think that, but your management might think failover means recovery to the point in time of the failure like the rest of us do, and could be a bit disapointed to discover that there are a bunch of messages that won't be processed until the queue manager comes back up. If one of those requests is to buy 10,000 shares of something, the delay could be very costly. If it's a request to buy $10 of left handed widgets, you could still get an irate customer on the phone if the fix takes a while.
If the order for $10 of left handed widgets is from a customer who buys $1m of parts from you over the year, and he goes to someone else who can process orders in a timely fashion, again possibly costly.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| WMBDEV1 |
| Posted: Thu Jul 09, 2009 12:38 am Post subject: |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
| aditya.aggarwal wrote: |
I think this desing is fulfilling the HA and Failover requirement for request -reply messages.
|
How are the marooned messages on the downed QM either HA or failed over? Bringing the downed QM as quick as you can is not a valid failover or HA solution in my opinon or do you sell every application you ever build as offering both HA and failover support because you'll presonally bring the affected application back as quick as you can?
Last edited by WMBDEV1 on Thu Jul 09, 2009 12:59 am; edited 1 time in total |
|
| Back to top |
|
|
| aditya.aggarwal |
| Posted: Thu Jul 09, 2009 12:46 am Post subject: |
|
|
Master
Joined: 13 Jan 2009
Posts: 252
|
Thanks Vitor...
I agree that in situaltion listed below my proposed design will not work...for this kind of requirement we need to take help of O/S Cluster e.g.: Sun Cluster, MSCS, HACMP etc...
This design will not work when the application used by the customers outside of the organization for critical transactions, orders, invoices etc....
where a delay may have impact on business.
my proposed design works for the HA and failover for the internal application of the company where the internal employee of the company are sending requests to get some statistics or data from other applications and they can wait for some time to get the reply back...
I was just trying to prove here that by using MQ Cluster with Clone queue managers we can acheive HA and failover at some extent...
Cheers,
Aditya |
|
| Back to top |
|
|
| aditya.aggarwal |
| Posted: Thu Jul 09, 2009 12:57 am Post subject: |
|
|
Master
Joined: 13 Jan 2009
Posts: 252
|
WMBDEV1 wrote:
| Quote: |
[quote="aditya.aggarwal" ]
I think this desing is fulfilling the HA and Failover requirement for request -reply messages.
|
How are the marooned messages on the downed QM either HA or failed over? Bringing the downed QM as quick as you can is not a valid failover or HA solution in my opinon or do you sell every application you ever build as offering both HA and failover support because you'll presonally bring the affected application back as quick as you can?[/quote]
No marooned message on downed qm will not failover or HA...
As i told alreaddy this design is applicalbe only when
1. Multiple requestors are requesting the same application 'XYZ'
2. Requestors can wait for the reply message
3. There will be no business impact if request message lost.
4. if one application , MQ server of 'XYZ' goes down then other server should be avialable to send reply to avoid any delay for new requests...
etc...
HA and failover design completely based upon the management requirement and above design is fulfilling the requirement of our management... 
Cheers,
Aditya |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Jul 09, 2009 1:05 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| aditya.aggarwal wrote: |
This design will not work when the application used by the customers outside of the organization for critical transactions, orders, invoices etc....
where a delay may have impact on business. |
No one inside your organisation performs critical transactions? Some of your business users may disagree with you!
| aditya.aggarwal wrote: |
| my proposed design works for the HA and failover for the internal application of the company where the internal employee of the company are sending requests to get some statistics or data from other applications and they can wait for some time to get the reply back... |
What your proposed design implies is that you have one WMQ setup for internal use by non-critical applications, and one with proper HA for critical use. Not only does that double your administration (and possibly license costs) but exposes your business to a large risk. Suppose some application designer or less experienced administrator uses the "non-critical" set up for a critical application because they don't know or don't understand the difference? Clearly you'd have some standards or controls to prevent this, but those standard & controls would be unnecessary in a more conventional set up.
| aditya.aggarwal wrote: |
| I was just trying to prove here that by using MQ Cluster with Clone queue managers we can acheive HA and failover at some extent... |
The design you're proposing gets trotted out on the forum every so often and always fails on the "stranded message" problem. The extent to which it works is dependant on the volume of message traffic and hence the risk there will be an unprocessed message on the queue at time of failure, combined with the management attitude to that risk. It's also reliant on these factors never changing in the future.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
|
  |
Goto page 1, 2 Next |
Page 1 of 2 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
