| Author |
Message
|
| zrux |
 Posted: Thu Apr 09, 2020 5:46 am Post subject: Multiple Gateway QMs in cluster |
 |
|
Apprentice
Joined: 21 May 2006
Posts: 37
Location: UK
|
We got 2 Datacenters
We are looking to have MQ ACTIVE/ACTIVE HA using the MQ clustering
We got several vendors which we don't want to make it part of the same cluster and they also wouldn't want to be part of our cluster. Hence thinking of creating a single cluster instead of overlapping cluster.
Request messages will flow inside the cluster using Gateway QM (QM to QM) and responses will go back to the cluster using the Gateway QM (QM to QM).
But how do I get around the same named Gateway QM being in 2 DCs. Cluster doesn't like same name QMs although one can argue that QMIDs will be different.
Is there a reference model/topology for doing this please.
Would the Gateway QMs are names different in the 2 DCs (GWQM1, GWMQ2) and I created a QM alias for each other on GWQM1, GWQM2 ..will that work ?
Any other way around this pls. |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Thu Apr 09, 2020 6:31 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
Have you tried using a different cluster receiver on each? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| hughson |
| Posted: Thu Apr 09, 2020 7:47 pm Post subject: Re: Multiple Gateway QMs in cluster |
|
|
Padawan
Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand
|
| zrux wrote: |
| But how do I get around the same named Gateway QM being in 2 DCs. |
Why do you have the same named Gateway QM ? NEVER design something that requires queue manager's to have the same name.
Please tell us the reason that is part of your design and we can provide alternates for that issue.
You just CANNOT run two different queue managers with the same name in a cluster. Only ONE OF THEM will be used. The cluster will assume the newest one is a delete + redefine of the old one and ignore the oldest one.
Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| zrux |
| Posted: Fri Apr 10, 2020 7:45 am Post subject: |
|
|
Apprentice
Joined: 21 May 2006
Posts: 37
Location: UK
|
1) not sure the use of different cluster receiver channel. How does that help in this case..can you provide some info
2 The requirement is, the QMs belonging to 3rd party needs to connect to the GW QMs in the 2 DCs to send and receive messages into and out of the cluster. How to resolve that..create a QM alias(with same QM name) for the 2 QMs(with different name QM name) on the GW and have the same named SDR/RCVR ..and create a script to reset the sequence number errors?
Is there any other way ?
In short has anyone implemented multiple QM acting as GW QM for putting and getting messages out of a cluster for QM to QM communications without being part of the cluster. |
|
| Back to top |
|
|
| hughson |
| Posted: Mon Apr 13, 2020 1:35 am Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand
|
| zrux wrote: |
| In short has anyone implemented multiple QM acting as GW QM for putting and getting messages out of a cluster for QM to QM communications without being part of the cluster. |
What you describe here is a normal and regularly seen setup. One queue manager in the cluster is setup as a gateway into the the cluster for queue managers belonging to other organisations that cannot/should not be members of the cluster. Normal (aka non-cluster) channels are setup between the gateway queue manager and the external queue manager, and with a small number of queue manager alias definitions the external queue managers can send messages to any queue in the cluster.
Here are the definitions to allow an external queue manager to connect into the cluster via a gateway.
On External Queue Manager:
| Code: |
DEFINE QLOCAL(GWQM) USAGE(XMITQ)
DEFINE CHANNEL(PARTY1.TO.GWQM) CHLTYPE(SDR) XMITQ(GWQM) CONNAME(...)
DEFINE CHANNEL(GWQM.TO.PARTY1) CHLTYPE(RCVR)
DEFINE QREMOTE(CQM) RNAME(' ') RQMNAME(CQM) XMITQ(GWQM) |
On Gateway Queue Manager:
| Code: |
DEFINE QLOCAL(PARTY1.XQ) USAGE(XMITQ)
DEFINE CHANNEL(GWQM.TO.PARTY1) CHLTYPE(SDR) XMITQ(PARTY1.XQ) CONNAME(...)
DEFINE CHANNEL(PARTY1.TO.GWQM) CHLTYPE(RCVR)
DEFINE QREMOTE(CQM) RNAME(' ') RQMNAME(' ') |
The important part in the above definitions is the QREMOTE called CQM. It is defined on the External Queue Manager to point to the Gateway Queue Manager, but then when you arrive on the Gateway Queue Manager, it is defined again to have a blank RQMNAME (Remote Queue Manager Name) which means it can go to any queue manager in the cluster. Both of these definitions named 'CQM' must have the same name. So if you change the name, ensure you change it on both queue managers.
What makes your setup rather more unique, and therefore complicated, is that you have a situation that breaks the cardinal rule: Don't have queue managers with the same name. As you have rightly intimated in your post:
| zrux wrote: |
| Have the same named SDR/RCVR ..and create a script to reset the sequence number errors? |
having channels with the same name, and transmission queues with the same name, will lead to problems.
So to solve this, for each external queue manager that has the same name, we will invent a name to refer to it in your environment. In the picture below, we have two external queue managers both named 'EXTQM'. We will refer to them as PARTY1 and PARTY2. Their channel names use the invented name, so therefore on the Gateway Queue Manager you have two distinct channels, and two distinct transmission queues, and no need to do unnatural things like reseting sequence number errors.
The definitions on each of the External Queue Managers are those shown above, with PARTY1 replaced with PARTY2 for the second external queue manager.
In order for an application running connected to one of the External Queue Manager to put to the Cluster Queue 'CQ1', it must use a fully qualified MQOPEN of CQ1 on CQM.
----
The more difficult part of the set up comes with the reply route. You said you needed to send and receive messages into and out of the cluster, so I assume you do need the reply route. Here's where the difficulty comes because you need to be able to address two queue managers with the same name, and be able to tell them apart. To get around this identity problem you must make use of reply-to-queue aliases. These are a special type of queue manager alias and the definitions are below and go with this second picture.
In order to hide the real queue manager name, the applications on the External Queue Manager must use the QREMOTE name 'REPLYQ.ALIAS' as the reply-To-Q name they put in the MQMD. But they must open and get from the QLOCAL reply queue. This may mean that the application needs to be recoded slightly. The second QREMOTE definition maps the invented name back to the real queue manager name now that a reply message has made it back to the correct queue manager.
On External Queue Managers:
| Code: |
DEFINE QLOCAL(REPLYQ)
DEFINE QREMOTE(REPLYQ.ALIAS) RNAME(REPLYQ) RQMNAME(PARTY1)
DEFINE QREMOTE(PARTY1) RQMNAME(EXTQM) |
When an application reading a message from a cluster queue, CQ1, sends a reply back, it will do an MQPUT1 to REPLYQ on PARTY1 (or PARTY2). The Gateway Queue Manager advertises the following QREMOTE definitions in the cluster, so that any queue manager in the cluster will know what to do with messages for queue managers they think are called PARTY1 or PARTY2.
On Gateway Queue Managers:
| Code: |
DEFINE QREMOTE(PARTY1) RQMNAME(PARTY1) XMITQ(PARTY1.XQ)
DEFINE QREMOTE(PARTY2) RQMNAME(PARTY2) XMITQ(PARTY2.XQ) |
Note: Now we see why the earlier transmission queue definitions on the Gateway queue manager didn't use the usual pattern of the queue manager name for the transmission queue name. Both QREMOTE and QLOCAL definitions are in the same namespace, so we couldn't make this QREMOTE definition if we already had a transmission queue of this name!
----
I have tested this setup before posting this, as queue manager aliases and reply-to-queue aliases are rather fiddly and I didn't trust myself to write this up just from memory. If you follow these instructions exactly and it doesn't work, I guess it means I have made a typo when translating it into this post, as this does work for me. Shout if you have problems. I have double checked for typos, but I can check again.
P.S. If I have misunderstood your setup, perhaps you can supply a picture to make it clearer?
Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| zrux |
| Posted: Mon Apr 13, 2020 2:38 pm Post subject: |
|
|
Apprentice
Joined: 21 May 2006
Posts: 37
Location: UK
|
Thanks Morag for the detailed explanation and giving valuable time to this, I appreciate very much.
However this topology poses a problem of single point of failure( for GWQM on Datacenter 1) when external QMs want to connect the MQ cluster using GWQM.
At present the design you have sent, the external QMs connect to 1GWQM providing HA on 2 cluster Qs for each of its Datacenters(DCs).
The requirement I had was to have the external QMs connecting to 1 GWQM on each datacentre. (see my point 2) in my previous post so as to get over single point of failure when connection from Exernal to GWQM is lost to DC1 GWQM.
Is there any other solution to this apart from the one I mentioned on point2.
Please do let me know if you this is not clear and you want a diagram.
[/img] |
|
| Back to top |
|
|
| hughson |
| Posted: Mon Apr 13, 2020 7:20 pm Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand
|
| zrux wrote: |
| However this topology poses a problem of single point of failure( for GWQM on Datacenter 1) when external QMs want to connect the MQ cluster using GWQM. |
Often Gateway queue managers are Multi-Instance Queue Managers, to allow them to be highly available, but without the complication of two alternate gateways into a cluster.
| zrux wrote: |
| Please do let me know if you this is not clear and you want a diagram. |
Yes please
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Apr 14, 2020 4:10 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| hughson wrote: |
| zrux wrote: |
| However this topology poses a problem of single point of failure( for GWQM on Datacenter 1) when external QMs want to connect the MQ cluster using GWQM. |
Often Gateway queue managers are Multi-Instance Queue Managers, to allow them to be highly available, but without the complication of two alternate gateways into a cluster. |
Or are otherwise made highly available using OS-level or similar technologies, where the same queue manager can be quickly (seconds not minutes) recovered.
The point (which I fully endorse while genuflecting towards @hughson) is that the solution to a queue manager being a single point of failure is NEVER that you have another identically named queue manager somewhere else in your topology.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Tue Apr 14, 2020 3:30 pm Post subject: |
|
|
Jedi
Joined: 25 Mar 2003
Posts: 2494
Location: Melbourne, Australia
|
| Vitor wrote: |
| ..Or are otherwise made highly available using OS-level or similar technologies, where the same queue manager can be quickly (seconds not minutes) recovered... |
Yes. OS-level high availability also provides a HA network address / host name. The external partners' qmgrs only need to know one network address.
We have one HA gateway qmgr that is used by all external partners. No unplanned outages in the last 10+ years, except for the occasional network glitch.
_________________
Glenn |
|
| Back to top |
|
|
| zrux |
| Posted: Wed Apr 15, 2020 4:11 am Post subject: |
|
|
Apprentice
Joined: 21 May 2006
Posts: 37
Location: UK
|
The requirement is both the Datacenters needs to be actively receiving traffic and without any single point of failure.
With the GWQM in DC1 getting the traffic and loadbalancing the request to QMs on DC1 and DC2 with queue clustering , there could be a case when DC1 crashes due to issues like Network has failed between external and DC1, NFS for MQ MI is corrupted and external partners cannot connect to DC1 etc
Hence the traffic needs to be able to go to DC2 as well. DC1 and DC2 needs to receive active/active from external QMs independently of each other.
Sorry not able to add an diagram so far..not able to find a site which can host the image |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Apr 15, 2020 4:33 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| zrux wrote: |
| The requirement is both the Datacenters needs to be actively receiving traffic and without any single point of failure. |
Then you're back to @hughson's point about queue manager alias. Your external partners need to connect to an alias name which maps to the queue manager actually running in DC1 or DC2.
Here's a question, which will I believe will help both you and ourselves think about this a little deeper.
So, you have an external partner connected to a gateway in DC1. This puts messages into a cluster and load balances over DC1 and DC2 queue managers. Awesome, ticks away like a Swiss watch, rainbows and unicorns dance the polka on the roof.
The gateway in DC1 has a failure as you describe. NFS goes wibble, darkness descends and a local man starts selling unicorn burgers from a food truck.
How, in your scenario, does the external partner switch to the other gateway? Does he have a DNS alias his end (because even if the gateway in DC2 has the same name it has a different IP address and his sender channel is pointing to DC1)? If so, what is the mechanism for them doing a stop / start to pick up the new address under the alias?
Remember that at his end, his sender channel has resolved the target address to the DC1 gateway and that's cached in the IP stack where MQ can't get to it.
So how does that work? How does that align with possible topology solutions that are being suggested here?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| zrux |
| Posted: Fri Apr 17, 2020 9:11 am Post subject: |
|
|
Apprentice
Joined: 21 May 2006
Posts: 37
Location: UK
|
the external partner in this case needs to treat DC1, DC2 GWQMs as MQ MI instance even thought its is not (1 IP for each DC using the same SDR channel and using a connection list for both the IPs).
When the external partner is not able to connect to DC1, the SDR channel will try to connect to DC2. It will get a sequence number error trying to send a persistent message. The DC1,DC2 will run a script every min to check for sequence errors and reset if needed. The SDR from DC will also check for sequence number error and needs to do a reset.
But my question is there a better way to do this?
Has anyone implemented a cluster (active/active) on 2 DC similar to this? |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Apr 17, 2020 11:16 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| zrux wrote: |
| The DC1,DC2 will run a script every min to check for sequence errors and reset if needed. The SDR from DC will also check for sequence number error and needs to do a reset. |
If the script is just detecting an error and blindly issuing a reset to clear the error, that's a great way to lose messages. As you make a point of saying these are persistent, I'm guessing this will make people cross.
| zrux wrote: |
| But my question is there a better way to do this? |
All the suggestions in this thread.
| zrux wrote: |
| Has anyone implemented a cluster (active/active) on 2 DC similar to this? |
My current employer has a gateway that links us to external partners and may or may not be forwarding messages into a cluster. This gateway can run in any of our data centers but from the outside appears to be the same queue manager with the same IP. So sender channel sequence errors are handled as they would be for any queue manager to queue manager connection.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| hughson |
| Posted: Fri Apr 17, 2020 5:31 pm Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand
|
| zrux wrote: |
| Sorry not able to add an diagram so far..not able to find a site which can host the image |
Feel free to send it to me directly - my email address can be found on https://www.mqgem.com/about.html I will host and display it here.
Alternatively, you could describe where my diagram is incorrect.
For example, I fear that your same named queue managers may not be where I have placed them? Could you describe which queue managers in your set up are the ones with the same name?
Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| zrux |
| Posted: Sat Apr 18, 2020 10:13 am Post subject: |
|
|
Apprentice
Joined: 21 May 2006
Posts: 37
Location: UK
|
1) if the reset is done looking at the sequence number in the error logs, i don't think it should lose the message.
2) I have sent mail to Morag with the diagram and bit more info. |
|
| Back to top |
|
|
|
|
