| Author |
Message
|
| George Carey |
 Posted: Wed Oct 10, 2007 3:35 pm Post subject: Highly Available intersite message traffic |
 |
|
Knight
Joined: 29 Jan 2007
Posts: 500
Location: DC
|
Looking for details on Best Practices when using IBM WSMQ for Intersite connectivity to attain highly available message traffic between the two sites. The two(2) primary goals of this connectivity are :
1.) message traffic continues to flow even with a single point failure
2.) failed server can be recovered with its stranded messages in timely fashion(timely is <= 15 minutes)
Two options come to mind :
1.) Use z/OS and shared MQ queues with multiple sender/receiver channels on both sites as the connecting message service providers. (non-Unix environment)
2.) Use two gateway servers at each site A1 and A2 and B1 and B2 say and have the same site servers host queue managers that are full repositories for WSMQ Clusters on each end. That is
..................................... A1 ................... B1
.............A_CLUSTER ...... | ..................... | ....... B_CLUSTER
.................................... A2 ................... B2
Now make the following additional MQ Cluster configurations:
A1_QMGR and A2_QMGR partial repos members of B_CLUSTER And symmetrically make
B1_QMGR and B2_QMGR partial repos members of C_CLUSTER
Now make A servers and B servers individually highly available with say VCS or HACMP and assuming we have ironed out the firewall issues between sites we are done, aren’t we ?
I remember reading (somewhere) that it is NOT recommended to have MQ Cluster channels between sites. They should be used in intrasite communication to affect ‘Higher’ application availability and application load balancing.
Questions:
Q1) Is that admonition correct? (no intersite MQ Clusters)… If so why?
Q2) What are the Best Practices for effecting the two goals stated above with WSMQ.
Likely issues would be: How are the cross site admin issues to be handled. Can site A admin servers at site B or vice-versa ? Enough for now … I am going on too much.
Anyone know of White paper or manual or book or ??? that properly addresses these issues. IBM manuals do not. And no support pac doco does either!
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding") |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Wed Oct 10, 2007 3:44 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Usually it is highly recommended to have all of a cluster within the same firewall / secure network. Problems with clusters and DNS and firewall + NAT + SSL are the stuff war stories are made of...
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| George Carey |
| Posted: Thu Oct 11, 2007 7:05 am Post subject: |
|
|
Knight
Joined: 29 Jan 2007
Posts: 500
Location: DC
|
Yes, exactly!! Looking to get info to address my question:
Looking for details on Best Practices when using IBM WSMQ for Intersite connectivity to attain highly available message traffic between the two sites. ...
Regards..
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding") |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Oct 11, 2007 7:25 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
What you can do, what you should do, and what works best... is going to really depend on the nature of the relationship between the two sites.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| George Carey |
| Posted: Thu Oct 11, 2007 7:51 am Post subject: Best Practices |
|
|
Knight
Joined: 29 Jan 2007
Posts: 500
Location: DC
|
What you can do, what you should do, and what works best...
Looking for some options on these three Whats. Ideally something Hursley might have put out on how to best address intersite connectivity with MQ. I put this under Clustering forum but it really is a generic MQ questions. I gave in about 4 sentences (for anyone who knows MQ) a good architecture for highly available connectivity (at no charge)! It is best for intrasite connectivity but could probably be used (with much more pain likely) for intersite connectivity. Just trying to get other good ideas (ideally best practices) at same price. As I say any written material that could be pointed to or 4 sentences with equally concrete examples as mine.
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding") |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Oct 11, 2007 8:06 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I guess I'm saying that the options aren't anything other than "all of the options for normal MQ communications".
And which particular set of choices is made has to be made based on specific knowledge of the specific relationship between the two sites.
It does no good to suggest a multi-layered cluster/gateway architecture when the business relationship between the two sites and the comparitive business needs of the two sides requires MQ clients run over MQIPT.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| George Carey |
| Posted: Thu Oct 11, 2007 8:30 am Post subject: Choose ones environment |
|
|
Knight
Joined: 29 Jan 2007
Posts: 500
Location: DC
|
Ok... I am happy to play that game to get some options. Choose/setup/describe your environment...
| Quote: |
| ..the business relationship between the two sites and the comparitive business needs of the two sides requires MQ clients run over MQIPT. |
... because:
Reason 1.) Details ... why and how ...
Reason 2.) Details ... why and how ...
...
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding") |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Oct 11, 2007 9:44 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
I don't know of any doc that talks about this specifically.
If it were me, I would design this as 2 seperate clusters at each company. Each company nominates 1 of their QMs as a Gateway QM to the other company. Those 2 QMs need to be made highly available (hardware clustering). They are connected to each other with regular SNDR / RCVR channels. Set the MCA on your CLUSRCVR to User1. Use setmqaut to grant User1 access only to Alias queue defined on the gateway QM, so they other side can only send messages to queues you allow.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| George Carey |
| Posted: Thu Oct 11, 2007 10:09 am Post subject: Now were talkin.. |
|
|
Knight
Joined: 29 Jan 2007
Posts: 500
Location: DC
|
Thank you Peter ... this is the kind of feedback I am looking for... isn't it interesting that I stumbled on some old LISTSERV archive on this same subject of intersite connectivity(2004)... with the best feedback and set of cautionary warnings to look into coming from non-other than yourself:
http://www.mail-archive.com/mqseries@akh-wien.ac.at/msg13899.html
On your suggested nominated gateway ... what do you think about two nominated gateways that would allow continuous message traffic for new messages should one fail (even for HA recover time of <= 15 minutes). These two gateways would be plugged into back end procesing MQ cluster as partials, to effect auto redirection of message traffic if one fails (no single point of failure).
Regards,
GTC
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding") |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Oct 11, 2007 10:21 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
MQ Clusters want to be fully connected networks. This is the problem with making any queue manager any kind of member of a cluster that is separated by a significant network boundary from the other members of the cluster.
If you want to use cluster channels to connect two sites - and you have considered all of the implications of doing this including the security implications - then you still want to separate the network boundary clustering from the rest of your internal cluster using regular MQ sender/receiver channels and qremotes and qaliases.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Oct 11, 2007 10:31 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
2 gateways on each side would be cool, so that if one goes down the other still takes traffic.
However, I can't think of a way to make my 2 gateways talk to their 1 or 2 gateways using regular SNDR / RCVR channels in such a way that if one of my gateways goes down the other one picks up all the the traffic. Remember that they will have q remotes and QM aliases on their side resolving to the XMITQ to your gateway. To only ONE XMITQ. So if that gateway goes down on your side your other one may be up, but so what. All their messages are still trying to go down the channel to the downed gateway.
Options include them scripting all the necessary actions anytime their SNDR channel to your Gateway#1 goes retrying (repoint the alaises and remote defs to use Gateway#2.XMITQ, moving stranded messages from Gateway#1.XMITQ to Gateway#2.XMITQ, etc). UGH! Properly setup you 1 highly available gateway QM should failover in 1-3 minutes, not 15. That's gotta be faster.
Or you get funky with 3 overlapping clusters. They have 2 gateways in THEIR.CLUSTER. You have 2 gateways in YOUR.CLUSTER. You then connect the 4 gateways in OUR.CLUSTER. Never tried it. Might not work. But would be fun to try. And a nightmare to debug when (not if) things go wrong.
I've thought about this topic off and on for years. I haven't come up with anything better than what I originally posted.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jsware |
| Posted: Thu Oct 11, 2007 11:40 am Post subject: |
|
|
Chevalier
Joined: 17 May 2001
Posts: 455
|
| PeterPotkay wrote: |
| 2 gateways on each side would be cool, so that if one goes down the other still takes traffic. |
We do exactly what Peter says earlier in this post to limit remote partners access to our queues. However (and I haven't explored this beyond my initial thoughts):
Have a highly available pair of hosts on each side of the firewall; MYHOSTA, MYHOSTB and URHOSTA, URHOSTB.
Have a qmgr on each host (MYQMA, MYQMB, URQMA, URQMB).
Make it so that MYQMA/B can run on both MYHOSTA/B using the HA supportpacs. Do the same with URHOSTA/B (URQMA/B run on both).
Now have *three* clusters; MYQMA/B & MYINTERALQMs are in MYCLUSTER, URQMA/B & URINTERNALQMs are in URCLUSTER.
Make just MYQMA/B and URQMA/B in OURCLUSTER (using name-lists).
Use the relevant QALIAS/QREMOTE definitions that are members of MYCLUSTER/URCLUSTER/OURCLUSTER so that a message put on MYINTERNALQMs get delivered, via OURCLUSTER to URINTERNALQMs as appropriate.
Then only the 4 qms in OURCLUSTER would need configuring on the firewall to allow all 4 to communicate (which would be the case if you wanted MYQMA/B to talk to URQMA/B in any combination using standard channels).
You can add more internal qms to MYCLUSTER or URCLUSTER as MYQMs are not affected by URCLUSTER and URQMs are not affected by MYCLUSTER.
Remember to use BIND NOT FIXED on the Q definitions. I would draw a diagram or 2 first!
_________________
Regards
John
The pain of low quaility far outlasts the joy of low price. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Oct 11, 2007 12:16 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
In addition, John's suggestion prevents MQ administrators on the other side from having any access to your internal clusters that you do not specifically grant or provide routes to. This is much harder to ensure if you overlap the three clusters.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Oct 11, 2007 12:31 pm Post subject: |
|
|
Guest
|
| Quote: |
| highly available message traffic between the two sites. |
Am I confused?
You mention z/OS MQ shared-queues. Multiple qmgrs sharing a shared-queue on a CF (coupling facility) satisfies message and qmgr failover - unless you have multi-message affinity and/or different applications).
By two sites , you mean that the z/OS site is one; and the other is not z/OS? Or is z/OS? If both are z/OS, are you Parallel Sysplex? For z/OS-to-z/OS, take a look at GDPS (Geographically Dispersed Parallel Sysplex). |
|
| Back to top |
|
|
| George Carey |
| Posted: Thu Oct 11, 2007 1:06 pm Post subject: Good stuff |
|
|
Knight
Joined: 29 Jan 2007
Posts: 500
Location: DC
|
Scottj2512, I have actually configured a similar 3 cluster overlapping setup and it works in theory. I used three clusters at each (simulated) site. 1.) a backend APP_CLUSTER 2.) a gateway GW_CLUSTER 3.)a site connecting OUR_CLUSTER. I had two servers in GW_CLUSTER with QM on each as you and Peter both describe. I plugged GW_QM1 and GW_QM2 into APP_CLUSTER as partials and had remote (simulated) site tied together as an OUR_CLUSTER as you describe. If I needed to pass messages into or out of a cluster I used QRemotes or QAlias as appropriate and namelisted them into one or more clusters as needed to make the message move to desired target again as you describe. But as I say this was simulated just to see if the thought processes were correct. It was all on one site without going through firewalls, etc. . . . I did not get buy in from partner site to actually try testing. They would only go sender/receiver chls between sites. It could be possible admin nigthmare as well if (when) something gets Cluster f...ed up. So Peter's options were basically what I was entertaining. We actually did a sdr/rcvr option test similar to the one Peter described where a custom program was written to monitor the channel and restart it using same Xmit q to new QM if channel failure detected. (This was at remote z/OS site and QM coming in via SDR/RCVR to our Clustered gateway qms.)
Did you say this was an actual setup of yours with Clustering between sites or was this initial thoughts? What Cluster channels between sites main benefit is the auto detection of channel failure and rerouting of new messages. This could be handled with SDR/RCVR channels if used in conjunction with some monitoring tool like Veritas or the like. Any one done anything in that space.
_________________
"Truth is ... grasping the virtually unconditioned",
Bernard F. Lonergan S.J.
(from book titled "Insight" subtitled "A Study of Human Understanding")
Last edited by George Carey on Thu Oct 11, 2007 1:59 pm; edited 3 times in total |
|
| Back to top |
|
|
|
|
