| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Clustering through a firewall |
« View previous topic :: View next topic » |
| Author |
Message
|
| mverh |
 Posted: Fri Mar 05, 2004 5:01 am Post subject: Clustering through a firewall |
 |
|
Voyager
Joined: 06 Mar 2002
Posts: 97
|
I have the following config:
CUST_QM outside a firewall.
REPOS_QM inside the firewall.
APP_QM inside the firewall.
REPOS_QM hosts CUST cluster and APP cluster.
CUST_QM is in the CUST cluster and APP_QM is in the APP cluster.
The firewall is configured to allow the CUST_QM and REPOS_QM to communicate using standard clustering.
A queue APP_Q in the CUST cluster is defined on the APP_QM. As a result channels are created that will enable CUST_QM to talk to APP_QM directly but these channels fail to start since the firewall is not configured to allow this.
My question is: Is it possible to have requests and replies flow from CUST_QM to APP_QM via REPOS_QM without having to define regular DQM channels?
I know I can create an alias queue on REPOS_QM that is in the CUST cluster that targets APP_Q. I did this and did get a message from CUST_QM to the APP_Q on APP_QM. But how do I get the reply from APP_QM back to CUST_QM w/o having to define a xmitq and/or a queue manager alias and regular DQM channels from APP_QM to REPOS_QM?
_________________
Marc Verhiel
IBM Canada Ltd. |
|
| Back to top |
|
 |
| oz1ccg |
| Posted: Fri Mar 05, 2004 5:48 am Post subject: |
|
|
Yatiri
Joined: 10 Feb 2002
Posts: 628
Location: Denmark
|
no problem, you're dealing with over-lapping clusters, so you define AliasQ on REPOS_QM for those queues that are "shared".
I have a example here:
http://www.mrmq.dk/cluster_qma_2.htm
CUST_QM: QME1
REPOS_QM: QMA1
APP_QM: QMA2
I hope this can help you to understand how it works, and make your configuration working aswell.
Just my $0.02 
_________________
Regards, Jørgen
Home of BlockIP2, the last free MQ Security exit ver. 3.00
Cert. on WMQ, WBIMB, SWIFT. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Mar 05, 2004 3:11 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7717
|
| Quote: |
REPOS_QM hosts CUST cluster and APP cluster.
CUST_QM is in the CUST cluster and APP_QM is in the APP cluster.
|
| Quote: |
A queue APP_Q in the CUST cluster is defined on the APP_QM.
|
These conflict? If APP_Q is in the CUST cluster, it can only be on CUST_QM or REPOS_QM, right? It can't be on APP_QM because APP_QM is not part of the CUST cluster. Maybe I am not understanding it correctly?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| oz1ccg |
| Posted: Sat Mar 06, 2004 1:09 am Post subject: |
|
|
Yatiri
Joined: 10 Feb 2002
Posts: 628
Location: Denmark
|
Peter,
I guess you're right
Lets take it again
APP_Q in APP cluster on APP_QM
CUST_Q in CUST cluster on CUST_QM
what we define on REPOS_QM is:
DEFINE QALIAS(APP_Q) CLUSTER(CUST) TARGQ(APP_Q)
DEFINE QALIAS(CUST_Q) CLUSTER(APP) TARGQ(CUST_Q)
This should do the trick, nice and easy.
just my $0.02
_________________
Regards, Jørgen
Home of BlockIP2, the last free MQ Security exit ver. 3.00
Cert. on WMQ, WBIMB, SWIFT. |
|
| Back to top |
|
|
| mverh |
| Posted: Sat Mar 06, 2004 9:34 am Post subject: |
|
|
Voyager
Joined: 06 Mar 2002
Posts: 97
|
The poblem does not lie with trying to get the request from the CUST_QM to the APP_Q on the APP_QM. Defining an alias "def qa(app_q) targq(app_q_on_app_qm) cluster(cust)" on REPOS_QM solves this. The problem is when the application on APP_QM consumes the request message and honours the reply to q information in the request MQMD. In this case queue name resolution occurs and since there is no queue named CUST_QM on APP_QM the response ends up on SYSTEM.CLUSTER.TRANSMIT.QUEUE and causes the channel from APP_QM to CUST_QM to try and start which fails due to the firewall.
I want the response to flow from APP_QM to CUST_QM via the REPOS_QM so I will need some sort of queue manager alias on APP_QM. But I don't think it is possible w/o regular DQM objects. |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Sat Mar 06, 2004 10:05 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2602
Location: The Netherlands (Amsterdam)
|
So what about defining a QManager ALIAS on APP_QM called CUST_QM which points to REPOS_QM ?
On APP_QM:
DEF QREMOTE(CUST_QM) RNAME() RQMNAME(REPOS_QM)
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sat Mar 06, 2004 11:16 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7717
|
Marc, no problem. I have 5 overlapping clusters, and all are talking to each other, sending request and replies back and forth.
The key is that on the gateway QM, the one QM that is in all the clusters (in your example just 2 clusters, in mine, 5), you define a Namelist. This Namelist contains the names of all the clusters.
So for you, make a Namelist, called CLUSTERNAMELIST, or whatever, whose names are "CUST, APP".
Now, on that gateway QM, you must define QM Aliases for every QM that is in these clusters, and cluster them to the namelist - very important that you cluster them to the namelist.
APP_QM has a QMAlias as follows:
DEF QREMOTE(APP_QM) RNAME() RQMNAME(APP_QM) CLUSNL(CLUSTERNAMELIST)
CUST_QM has a QMAlias as follows:
DEF QREMOTE(CUST_QM) RNAME() RQMNAME(CUST_QM) CLUSNL(CLUSTERNAMELIST)
Then you should have a QMAlias for every cluster name, like so:
DEF QREMOTE(CUST) RNAME() RQMNAME() CLUSNL(CLUSTERNAMELIST)
DEF QREMOTE(APP) RNAME() RQMNAME() CLUSNL(CLUSTERNAMELIST)
At this point, you are all set for sending messages from any QM to any other QM or cluster.
Lets take the example that an app on APP_QM wants to send a request to RequestQ on CUST_QM, and expects the reply to come back to ReplyQ on APP_QM.
Step one, we need to produce a request message that is properly addressed. We can either specify the Queue Name and Queue Manager name on the MQOPEN, or define a remote queue on APP_QM. The net result of either method is that a message is produced leaving APP_QM looking for RequestQ/CUST_QM.
Since the QMAlias CUST_QM is visible to APP_QM, the message makes it to REPOS_QM. Once the message is at REPOS_QM, it will flow to RequestQ on CUST_QM.
Note that if you wanted, you could have addressed the message as RequestQ/CUST, to take advantage of roundrobining. In this case, the message would make it from APP_QM to REPOS_QM (because APP_QM can see that QMAlias you defined for CUST). Once there, the QMAlias for CUST would have blanked out the QM name. At this point, the message destined for RequestQ no longer is looking for a specific QM, and thus roundrobining can take place.
Now more specifically to your question, the part of the reply.
The message made it into RequestQ. The app takes it, and is now ready to do a put to the Reply2 info in the header, which is ReplyQ/APP_QM.
It does the MQOPEN, and the message resolves to REPOS_QM, because you defined a QMALias called APP_QM that is visible to all the QMs in all the clusters (the power of cluster namelists). So the reply message makes it to REPOS_QM. There, it is still looking for ReplyQ/APP_QM. And it resolves to ReplyQ on APP_QM because REPOS_QM is in the same cluster as APP_QM.
There is no need for Q Aliases, just QM Aliases.
Properly set up, you can have any QM in any cluster talk to any other QM specifically, or any QM in any cluster talk to any other cluster. Requests and replies will all flow perfectly.
In summary, on your gateway QM:
1.Define a Namelist that contains all the names of all you clusters.
2. Define a QMAlias for every QM in all your clusters, and cluster those QMAliases with a ClusterNamelist attribute. Now all QMs in all clusters see these QMAliases, and so see all the other QMs.
3. Define a QMAlias for every cluster name, and cluster those QMAliases with a ClusterNamelist attribute. Now all QMs in all clusters see these as well.
4. When sending a request, properly address the message as DestinationQueue/DestinationQM, or DestinationQ/DestinationCluster. You do this by either defining a remote queue, or having the app build the MQOD with both the Q name and QM name.
5. The replies will take care of themselves, as the replying app will fully address the reply message using the Reply2Q and Reply2QM fields.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Sat Mar 06, 2004 11:25 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2602
Location: The Netherlands (Amsterdam)
|
Peter brilliant!!!
I never understood why CLUSTER and CLUSNL attributes existed on QREMOTE
now I do... 
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| oz1ccg |
| Posted: Sun Mar 07, 2004 2:53 pm Post subject: |
|
|
Yatiri
Joined: 10 Feb 2002
Posts: 628
Location: Denmark
|
Great job Peter  There are a million ways to configure a cluster configuration..... I guess the configuration depends on your buisness needs, your skills and our fantacy.
By the way how are you the dealing with security e.g. not showing off your whole cluster configuration to outsiders ??
And in the same way preventing eg. customers from sending messages to your internal queues ??
| Quote: |
| Properly set up, you can have any QM in any cluster talk to any other QM specifically, or any QM in any cluster talk to any other cluster. Requests and replies will all flow perfectly. |
I think  it's normally so (when crossing firewall area) we're dealing with other companies, and in this manner (due to legal stuff) I don't like to expose the whole cluster configuration. I just define the needed queues (ALIASQ) on the gateway queuemanager. And not give an intruder any chance to send a message to a "not to him autorized" queue.
Just my $0.02
_________________
Regards, Jørgen
Home of BlockIP2, the last free MQ Security exit ver. 3.00
Cert. on WMQ, WBIMB, SWIFT. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sun Mar 07, 2004 4:28 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7717
|
My solution (as laid out above) deals only with clusters who host apps where it is deemed OK that they can send to any queue on any QM. Marc did not mention any security requirements.
But if there were, and overlapping clusters with an external customer are needed (a big no no for me, see bottom of this post), you could do the following:
If CLUSTER1 is sending messages to some other QM or cluster, they must pass thru the common gateway QM (REPOS_QM, to use Marc's name). On REPOS_QM, you would have 5 separate CLUSRRCVs channels, one for each cluster:
TO.REPOS_QM.CLUSTER1
TO.REPOS_QM.CLUSTER2
TO.REPOS_QM.CLUSTER3
TO.REPOS_QM.CLUSTER4
TO.REPOS_QM.CLUSTER5
Each one is clustered only to the appropriate cluster.
If CLUSTER1 is the external one, you set the MCAUSER on TO.REPOS_QM.CLUSTER1 to be "STAYOUT_ID". Any messages that come from CLUSTER1 will arrive on REPOS_QM tagged with the ID "STAYOUT_ID". You give this ID access to the DLQ and the cluster transmit queue on REPOS_QM. Any message that comes thru from CLUSTER1 can now pass to any other QM in any of the other clusters. BUT, once on those QMs, the message will just go to the DLQ of that QM unless you allow STAYOUT_ID put access to the queues you allow, and only the queues that you allow.
The idea that you laid out here:
http://www.mrmq.dk/cluster_qma_2.htm
will certainly also do the trick. In your solution, you just have to create QALAISes for all the internal queues (these are so the external customer can see them), and QueueManagerAlaises only for the external QMs (this assumes that you dont care that your internal apps can put to any of the queues on the external QMs).
If there are security considerations, I would SERIOUSLY consider not overlapping clusters with an external customer. Let them have their cluster, you have yours, and connect their gateway QM to your EDGE queue manager with regular channels, which then connects to your gateway QMs with regular channels. (An Edge QM is one that sits at the edge of your infrastructure talking to some other company.)
Why use an Edge QM? Because if you really are worried about security, the QM that is exposed to the other company is very vulenrable to about 12 different attacks that have nothing to do with clustering. As a twisted bastard at the external customer, I can crash your QM (that talks to us directly) so fast its not funny, or create all sorts of MQ changes / additions / deletions to any and all (yes, ALL) of the QMs in all of your clusters. (Maybe I'll just start changing some of the queues to BIND_ON_OPEN on random QMs in your overlapping cluster, or maybe I'll PUT_INHIBIT your DLQs.)
Edge QMs can be protected, but it is restrictive. You don't want to make your gateway QM so locked down it makes life for you an administrative nightmare if it has to also do all the regular internal work. Much easier to lockdown a specified QM, the EDGE QM.
And, maybe I haven't thought of all the different ways that a QM can be crippled. What would you rather tell your boss, that the Edge QM is down, but all internal MQ traffic is OK, or that the gateway QM for 5 overlapping clusters just died  , and all your internal MQ cluster traffic is out?
_________________
Peter Potkay
Keep Calm and MQ On
Last edited by PeterPotkay on Sun Mar 07, 2004 5:33 pm; edited 1 time in total |
|
| Back to top |
|
|
| mverh |
| Posted: Sun Mar 07, 2004 5:19 pm Post subject: |
|
|
Voyager
Joined: 06 Mar 2002
Posts: 97
|
Peter, excellent design...this shows the power of clustering...the only thing I am concerned about is:
| Quote: |
4. When sending a request, properly address the message as DestinationQueue/DestinationQM, or DestinationQ/DestinationCluster. You do this by either defining a remote queue, or having the app build the MQOD with both the Q name and QM name.
|
I don't think I want to have to specify anything in the MQOD.ObjectQMgrName of the requesting app that exists on CUST_QM. I want this left blank so that the the servicing app on the APP_QM is portable (i.e. location independance). If the requester has to know about the APP_QM or the APP cluster then I wouldn't be able to move my servicing app w/o changing the requesting app on CUST_QM.
In your proposed solution what would happen if the requsting app on CUST_QM did not specify the MQOD.ObjectQMgrName and left it blank? Would it still work? |
|
| Back to top |
|
|
| oz1ccg |
| Posted: Mon Mar 08, 2004 2:17 am Post subject: |
|
|
Yatiri
Joined: 10 Feb 2002
Posts: 628
Location: Denmark
|
I can only agree with you Peter on the security; it's not a good idea to open clusters aginst "untrusted" parties, and yes you/I can do a lot of harm to an "open" cluster comunity. So Cluster are for INTERNAL use only. 
By the way, I'll try to find a way of extending my samples with some of the information addressed here. Thanks Peter.
Anyway, using two EDGE qmgrs with shared queues/channels or a HACMP solution for resillion purposes, that might be the answer.
Now it's funny to archive cluster workload balancing thru a EDGE qmgr, just using a "CLUSTER-ALIAS" and refer to that from the remote qmgrs. Nice and easy.
Just my $0.02
_________________
Regards, Jørgen
Home of BlockIP2, the last free MQ Security exit ver. 3.00
Cert. on WMQ, WBIMB, SWIFT. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Mar 08, 2004 4:48 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7717
|
| Quote: |
I don't think I want to have to specify anything in the MQOD.ObjectQMgrName of the requesting app that exists on CUST_QM. I want this left blank so that the the servicing app on the APP_QM is portable (i.e. location independance). If the requester has to know about the APP_QM or the APP cluster then I wouldn't be able to move my servicing app w/o changing the requesting app on CUST_QM.
In your proposed solution what would happen if the requsting app on CUST_QM did not specify the MQOD.ObjectQMgrName and left it blank? Would it still work?
|
I am going to assume the first sentence you meant to say "I don't think I want to have to specify anything in the MQOD.ObjectQMgrName of the requesting app that exists on APP_QM.
Just use Remote Queue Definitions. The app on APP_QM only needs to know the name of the remote queue def, which can be the same on many different queue managers, thus allowing your app to move between testing environments without worrying about the destination QM. Once the message resolves to the remote queue def, it will be properly addressed by the remote queue def. Remote Queue Name will = Desitnation Queue, Remote Queue Manager name will = Desitnation Queue Manager.
The MQ admin just makes sure the remote queue def is correct in each environment. The app just uses the same named remote queue def in all environments.
On the replying side, the app replies to the Reply2Q and Reply2QM, which will always be correct, so there is no MQ admin work there (after the QMAliases are set up that is).
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mverh |
| Posted: Mon Mar 08, 2004 6:22 am Post subject: |
|
|
Voyager
Joined: 06 Mar 2002
Posts: 97
|
| Peter, in my original post the requester is resident on the CUST_QM and the servicing app was on APP_QM. I think you transposed the two when explaining the architecture. Regardless, your architecture will work and I don't need to specify any value in the MQOD.ObjectQMgrName of the requesting app as long as I have a remote queue definition as you state. So I assume the qalias I mentioned in my original post that is resident on REPOS_QM is not needed when I implement your architecture? |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Mar 08, 2004 3:34 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7717
|
Correct.
If you have the QMAliases defined for all the QMs and all the cluster names, there is no need for QAliases, although it does not preclude you using them for other reasons.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
