| Author |
Message
|
| shehryar |
 Posted: Thu Mar 18, 2010 3:51 am Post subject: QM-QM connection for HA and Load Balancing |
 |
|
Novice
Joined: 11 Mar 2010
Posts: 14
|
Hi,
Scenario: Client -> local WMQ Server -> Central WMQ (clustered queues with two QMs). A gateway is also configured to communicate with other QMs outside the cluster.
For HA: The Central WMQ Server setup has been replicated in another Data Centre. The HA setup is Active-Active.
This results in: 4 QMs, 2 Gateways
Questions:
1. Since the HA cluster is active-active for Central WMQ, can I install multiple QMs on the local WMQ server and connect them with both gateways on Central+Backup WMQ? E.g. 2 QMs on local WMQ connect to 4 QMs on Central WMQs through 2 gateways. This achieves load balancing???
2. If one gateway goes down, the HA setup will failover to the next available gateway and queue cluster. In this case, only 1 QM will work on local WMQ...the one configured to work with currently available QM in Central WMQ. This achieves HA?
Does this design sound right? Or should we only connect to a single gateway at a time from multiple local QMs. If that gateway goes down, all
local QMs are the migrated automatically to the backup gateway.
Thoughts??? |
|
| Back to top |
|
 |
| shehryar |
| Posted: Wed Mar 31, 2010 4:57 pm Post subject: |
|
|
Novice
Joined: 11 Mar 2010
Posts: 14
|
Let's try it this way...
1. Central WMQ server: hardware clustered in active-active config across two data centre. One gateway QM configured in each data centre. MQ clustering for load balancing
2. Local WMQ server: no MQ clustering, hardware clustered for HA.
If a QM on Local WMQ server is configured to connect to the Gateway QM of Data Centre 1, is there an automatic option to failover to Gateway of Data Centre 2 if the connection with Data Centre 1 is not available any more? Or do I have to make QM on Local WMQ part of MQ cluster of Central WMQ to achieve this?
Regards |
|
| Back to top |
|
|
| mvic |
| Posted: Wed Mar 31, 2010 5:23 pm Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| Maybe client reconnect is what you are looking for? Or a client channel definition table? Or rely on virtual IP addresses forwarding your client's connection to the "live" machine? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Mar 31, 2010 9:36 pm Post subject: Re: QM-QM connection for HA and Load Balancing |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| shehryar wrote: |
Hi,
Scenario: Client -> local WMQ Server -> Central WMQ (clustered queues with two QMs). A gateway is also configured to communicate with other QMs outside the cluster.
For HA: The Central WMQ Server setup has been replicated in another Data Centre. The HA setup is Active-Active.
This results in: 4 QMs, 2 Gateways
Questions:
1. Since the HA cluster is active-active for Central WMQ, can I install multiple QMs on the local WMQ server and connect them with both gateways on Central+Backup WMQ? E.g. 2 QMs on local WMQ connect to 4 QMs on Central WMQs through 2 gateways. This achieves load balancing??? |
No. To achieve load balancing you typically use an MQ Cluster.
| shehryar wrote: |
| 2. If one gateway goes down, the HA setup will failover to the next available gateway and queue cluster. In this case, only 1 QM will work on local WMQ...the one configured to work with currently available QM in Central WMQ. This achieves HA? |
Does not sound quite right. You do not specify what you are using your gateway and other clustered qmgrs for...
| shehryar wrote: |
Does this design sound right? Or should we only connect to a single gateway at a time from multiple local QMs. If that gateway goes down, all
local QMs are the migrated automatically to the backup gateway.
Thoughts??? |
You sound like you are confusing HA Cluster and MQ cluster. They can both overlap but are used for different things.
A clear picture of what is running on which part of the HA cluster and how the MQ cluster looks will clarify this picture and help you understand...
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| shehryar |
| Posted: Thu Apr 01, 2010 1:27 am Post subject: |
|
|
Novice
Joined: 11 Mar 2010
Posts: 14
|
Thanks both for the response.
mvic: This is a server-server connection (local WMQ server to central WMQ server). Client connection is not being used here
fjb_saper: The customer uses central WMQ server which is configured for active-active HA setup across two data centres. MQ clustering is also configured for load balancing. Both data centres have Gateway QMs for connecting QMs outside the cluster.
To send data from local WMQ server to Central WMQ server, a local QM will be configured to connect with Gateway QM in first data centre. If for any reason, this connection goes down or the Gateway QM isn't available any more, how do I ensure that the local QM can continue to send data to the Gateway QM in data centre 2?
In the SAP world, we typically used external load balancing/switchover software/hardware to route traffic to any of currently available application servers. This is the sort of thing an MQ cluster will offer. So does this mean I have to include my local QM in the MQ cluster on Central WMQ to avail the capability of automatic failover to second node of the active-active cluster? Can I use something like DNS redirect? or any other mechanism to ensure local WMQ server can continue to work with either of the Gateway QMs in both data centres?
Thanks for your time. |
|
| Back to top |
|
|
| sumit |
| Posted: Thu Apr 01, 2010 3:42 am Post subject: |
|
|
Partisan
Joined: 19 Jan 2006
Posts: 398
|
| shehryar wrote: |
| If for any reason, this connection goes down or the Gateway QM isn't available any more, how do I ensure that the local QM can continue to send data to the Gateway QM in data centre 2? |
As you are using server-rqstr/receiver pair, the local queue manager will not redirect the traffic to Gateway qmgr in data center 2. You need to manually change the conn name so that your channel can connect to new gateway. More options discussed below.
| shehryar wrote: |
| So does this mean I have to include my local QM in the MQ cluster on Central WMQ to avail the capability of automatic failover to second node of the active-active cluster? |
Yes, if you need automatic switch over, then this is a better option. But may be you don't want the local qmgr to be same cluster with your gateway and internal qmgrs.
Please check cluster weight/ Rank options. It will help you.
| shehryar wrote: |
| Can I use something like DNS redirect? |
I think this will work for Server-Rqstr/Rcvr pair.
_________________
Regards
Sumit |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Apr 01, 2010 12:54 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| shehryar wrote: |
Thanks both for the response.
mvic: This is a server-server connection (local WMQ server to central WMQ server). Client connection is not being used here
fjb_saper: The customer uses central WMQ server which is configured for active-active HA setup across two data centres. MQ clustering is also configured for load balancing. Both data centres have Gateway QMs for connecting QMs outside the cluster. |
Solid design. Of course the Gateway is also part of HA...
| shehryar wrote: |
| To send data from local WMQ server to Central WMQ server, a local QM will be configured to connect with Gateway QM in first data centre. If for any reason, this connection goes down or the Gateway QM isn't available any more, how do I ensure that the local QM can continue to send data to the Gateway QM in data centre 2? |
That is what you use HA for. If gateway1 goes down in dc1 it will be brought up in dc2 (HA clustering). You should have a virtual IP fronting this so that no changes are required to your local qm and your qm to qm connection will just suffer the interruption for the time of the HA switchover.
| shehryar wrote: |
In the SAP world, we typically used external load balancing/switchover software/hardware to route traffic to any of currently available application servers. This is the sort of thing an MQ cluster will offer. So does this mean I have to include my local QM in the MQ cluster on Central WMQ to avail the capability of automatic failover to second node of the active-active cluster? Can I use something like DNS redirect? or any other mechanism to ensure local WMQ server can continue to work with either of the Gateway QMs in both data centres?
Thanks for your time. |
No. If your local QM connects to the Gateway QM (HA protected) that should be enough. The cluster will then take over load balancing once the message has reached the Gateway. Remember the Gateway qmgr is part of both the HA cluster(availability) and the MQ Cluster (load balancing).
Your HA software will provide a single virtual IP for the gateway qmgr. This is what you need to use both inside the MQ cluster and from outside to connect. The real IP is only used by the HA software to route the calls accordingly to the 'live' HA partition.
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| shehryar |
| Posted: Mon Apr 05, 2010 12:30 pm Post subject: |
|
|
Novice
Joined: 11 Mar 2010
Posts: 14
|
Thanks both for your response. This clarified the concern of Gateway 1 going down. HA software will fail it over to datacentre 2.
However, what about connection unavailability? If the connection to Gateway 1 is not available any more, how do I ensure that the local QM can continue to send data to Gateway 2? Does this require mandatory use of MQ clustering of local and central QMs if I want to avoid manual redirect using DNS/IP redirect?
Regards |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Apr 05, 2010 12:42 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| shehryar wrote: |
| However, what about connection unavailability? If the connection to Gateway 1 is not available any more, how do I ensure that the local QM can continue to send data to Gateway 2? |
Why not fail it over to datacentre 2 as you would if the machine crashed? Autmatically if the HA software's bright enough to spot a connection failure, manually if you're reliant on other monitoring.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| shehryar |
| Posted: Tue Apr 06, 2010 3:24 am Post subject: |
|
|
Novice
Joined: 11 Mar 2010
Posts: 14
|
Thanks Vitor. A failover to data centre 2 can only happen if the local QM can reach Gateway 1 which later crashed. The HA mechanism will work once you can reach the system. What about the case when the network is down and connection to Gateway 1 can't be established? This is where an external load balancing mechanism kicks in. The other option is to use MQ clustering to achieve the same to make sure Local QM can continue to send messages on an alternate network route...
Thoughts??? |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Apr 06, 2010 5:14 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| shehryar wrote: |
| What about the case when the network is down and connection to Gateway 1 can't be established? |
You manually tell the HA software that Gateway 1 is inaccessable and have it fail over to Gateway 2 (which presumably is still accessable in this scenario).
I apologise if I failed to properly articulate this in my last post.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Apr 06, 2010 8:41 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| shehryar wrote: |
Thanks Vitor. A failover to data centre 2 can only happen if the local QM can reach Gateway 1 which later crashed. The HA mechanism will work once you can reach the system. What about the case when the network is down and connection to Gateway 1 can't be established? This is where an external load balancing mechanism kicks in. The other option is to use MQ clustering to achieve the same to make sure Local QM can continue to send messages on an alternate network route...
Thoughts??? |
QMGR to QMGR
Well if the network is down (presumably you cannot get to any destination) the message will have to wait in the XMITQ until the long and short retrys are done (manual channel restart) or the connection is restored and a retry kicked in.
CLIENT
If the network is down you will not be able to connect or if connected get a failure. The application will have to handle this. If only parts of the network are down a channel table may allow you to connect to a qmgr that is on an accessible part of your network.
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| shehryar |
| Posted: Tue Apr 06, 2010 11:53 pm Post subject: |
|
|
Novice
Joined: 11 Mar 2010
Posts: 14
|
Thanks both for your response. Much appreciated.
The scenario I am looking at is if the local QMGR can't connect with Gateway 1 because network link is down. However, network link to Gateway 2 is still available. So local QMGR should be able to connect to Gateway 2 automatically without changing any configuration. It seems this is only achievable through MQ clustering.
Question: Does this mean additional configuration is required to ensure the local QMGR is used for processing local messages only? I think this could be achieved by having a partial repository on the local QMGR and excluding local QMGR from partial repositories on other QMGRs in the cluster. This should ensure local messages are only processed by local QMGR and aren't sent to an arbitrary QMGR in the cluster. Similarly, non-local messages won't arrive in local QMGR for processing. Does this sound right??? Or is there still a possibility that the local QMGR will be dragged into processing non-local messages?
Client connection is not an option since the link is trans-atlantic.
Regards |
|
| Back to top |
|
|
| exerk |
| Posted: Wed Apr 07, 2010 12:30 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| shehryar wrote: |
| ...The scenario I am looking at is if the local QMGR can't connect with Gateway 1 because network link is down. However, network link to Gateway 2 is still available. So local QMGR should be able to connect to Gateway 2 automatically without changing any configuration. It seems this is only achievable through MQ clustering... |
OK, logical so far...
| shehryar wrote: |
| ...Question: Does this mean additional configuration is required to ensure the local QMGR is used for processing local messages only?... |
Don't share the 'local' queues in any clusters...
| shehryar wrote: |
| ...I think this could be achieved by having a partial repository on the local QMGR and excluding local QMGR from partial repositories on other QMGRs in the cluster... |
I'm not sure how you could achieve this without writing something like an exit that allows only your Gateway queue managers DNS/IP Addresses to connect/be connected to...
| shehryar wrote: |
| ...This should ensure local messages are only processed by local QMGR and aren't sent to an arbitrary QMGR in the cluster... |
If the queue on the 'local' queue manager only exists in that queue manager, then no message will 'arbitrarily' go anywhere except to that 'local' queue...
| shehryar wrote: |
| ...Similarly, non-local messages won't arrive in local QMGR for processing... |
See above...
| shehryar wrote: |
| ...Does this sound right??? Or is there still a possibility that the local QMGR will be dragged into processing non-local messages?... |
As stated previously, if the queues aren't available in the cluster then other queue managers can't send messages to them...
If I understand your topology correctly, you could always build an overlapping cluster consisting of your 'local' queue managers and gateway queue managers, which will give your 'locals' access to all the central queue managers via load-balanced gateways.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Apr 07, 2010 5:10 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| exerk wrote: |
| If I understand your topology correctly, you could always build an overlapping cluster consisting of your 'local' queue managers and gateway queue managers, which will give your 'locals' access to all the central queue managers via load-balanced gateways. |
 From the questions of the OP it looks like this is the topology he is looking for.
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
