| Author |
Message
|
| kkumar_70 |
 Posted: Mon Feb 04, 2008 11:54 am Post subject: Partial Repository Communications |
 |
|
Novice
Joined: 04 Feb 2008
Posts: 10
|
All/Gurus,
I have a very simple yet very intriguing query. Here is the setup.
FR1 - Full repository
PA1 - Partial Repos 1
PA2 - Partial Repos 2
FR1 - Clus rcvr defined
PA1 - Clus rcvr and clus sdr defined pointing to FR1
PA2 - Clus rcvr and clus sdr defined pointing to FR1
CLUSQ - defined as a cluster queue on PA2
Channels are all up and running and all the 3 queue managers can view the CLUSQ.
So far so good.
Now, i try to put a message on CLUSQ connecting to PA1. Successfully put a message and everything is just fine. As one would expect from clustering.
The problem though is, i see an auto-define clussdr channel between PA1 and PA2. I do realize that within clustering auto-defined cluster channels are created on at-needed basis. However, i always thought that was the case between FR and PR* and NOT between PR* and PR*.
Documentation is hush-hush as it is not very clear around this query of mine. I have not been able to prove that this is wrong or otherwise. Neither have i gotten any justification as to why this should be the case.
If every PR* is able to connect to every other PR* then it is simple point-to-point DQM setup. What is the point of clustering then?
Moreover, i do see channels between PR1->FR->PR2 and PR1->PR2. If that is the case, if i put a message on a clustered queue hosted on PR2, which route it will follow.
Whole lot of confusions.
PS: I searched extensively on this forum and IBM site for anything closer to my query and couldn't find anything relating to this and hence this post.
Thanks in advance
KK |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Mon Feb 04, 2008 12:06 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
MQ Clusters will become fully connected networks.
More or less whether you want them to or not - because channels are autodefed to make direct connections.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| kkumar_70 |
| Posted: Mon Feb 04, 2008 12:09 pm Post subject: |
|
|
Novice
Joined: 04 Feb 2008
Posts: 10
|
| jefflowrey wrote: |
MQ Clusters will become fully connected networks.
More or less whether you want them to or not - because channels are autodefed to make direct connections. |
Doesn't that defeat the purpose of having so many definitions and then how would it differ from a regular P2P communication, other than the fact that in P2P you have to manually manage channels and in clustering it is done by the qm.
KK |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Feb 04, 2008 12:16 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
The fact that in p2p you have to manually manage the channels is exactly the point of MQ clustering.
Well, one of two main points.
The two main points of MQ Clustering are a) reduced administration and b) workload balancing.
The a) comes in because of the autodef. It means that if I have a network of 100 queue managers, I don't have to manually define 100! channels - just 200 (give or take).
The autodef channels between the FR and the PR are mainly for cluster definition propagation, unless your apps are sending messages to the FR?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| kkumar_70 |
| Posted: Mon Feb 04, 2008 1:00 pm Post subject: |
|
|
Novice
Joined: 04 Feb 2008
Posts: 10
|
Jeff,
Is there somewhere in the doc that you can point me to that states what you stated, which is Clustering does infact follows P2P under the covers.
Moreover, the confusion of debugging such a clustering setup would be huge. You would need to debug a message lost case between P2P as well as P1->FR->P2 scenario. This will duplicate everything and would create a mess if something does in fact goes wrong. Hence, i still do not see any advantage of Clustering following a P2P setup other than MQ managing the additional channels/overheads, which doesn't really advocate someone going to Clustering et all.
Communication between clustering participants should have ideally been flowing through the FR instead of a direct P2P IMHO.
Assume having a cluster with 1000 queue managers with a many-to-many setup. Clustering isn't really helping avoid the mess that P2P would have generated. If everything flowed through FR, then we would have a much cleaner QM with just 2 channels running at any point in time.
KK |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Feb 04, 2008 1:03 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Messages never go by an indirect route.
That's why the network becomes fully connected.
If a message is sent from Qmgr A and destined for Qmgr C, an MQ Cluster channel will be created from Qmgr A that connects to Qmgr C. If that channel won't start... the message won't flow.
I will suggest a review of the Using WMQ Cluster manual.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| kkumar_70 |
| Posted: Mon Feb 04, 2008 1:08 pm Post subject: |
|
|
Novice
Joined: 04 Feb 2008
Posts: 10
|
| jefflowrey wrote: |
Messages never go by an indirect route.
That's why the network becomes fully connected.
If a message is sent from Qmgr A and destined for Qmgr C, an MQ Cluster channel will be created from Qmgr A that connects to Qmgr C. If that channel won't start... the message won't flow.
I will suggest a review of the Using WMQ Cluster manual. |
Jeff, when i wrote the above statement, assumption was messages were flowing.
Let me put it this way. I am not posting a complaint nor am i in a situation that i need to fix. This post is just made for general discussion as i wanted to understand if someone else can interpret what i am unable to.
And yes, i have been associated with MQ long enough to have read manuals 100's of times. 
Question has been, where in the manuals does it say what you state. Which is Clustering does a P2P and it's benefits/disadvantages etc.
Why doesn't all messages flow through the FR rather than establishing a direct P2P connection is my question. I do understand that is how it works, but i am looking for some reasoning here. 
Cheers
KK |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Feb 04, 2008 1:18 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
|
| Back to top |
|
|
| kkumar_70 |
| Posted: Mon Feb 04, 2008 1:46 pm Post subject: |
|
|
Novice
Joined: 04 Feb 2008
Posts: 10
|
Thanks. But those links did look familiar to me. If there is anything else that you/others can add to the questions that i rasied, would be great. Otherwise, we can consider this query closed yet unresolved. 
| Quote: |
Why doesn't all messages flow through the FR rather than establishing a direct P2P connection is my question. I do understand that is how it works, but i am looking for some reasoning here. |
KK |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Feb 04, 2008 1:54 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I'm confused.
Those pages answer your question.
In particular
| Quote: |
| If QM1 wants to send some messages to queues at QM3, it automatically creates a cluster-sender channel connecting to the cluster-receiver channel at QM3. |
If the FR is across a long, slow WAN, and the second PR is at the next port on the same switch.... It's not clear why you would want messages to take a potentially very much longer route - by way of at least another hop to the FR - from QM1 to QM3.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| kkumar_70 |
| Posted: Mon Feb 04, 2008 2:05 pm Post subject: |
|
|
Novice
Joined: 04 Feb 2008
Posts: 10
|
| jefflowrey wrote: |
I'm confused.
Those pages answer your question.
In particular
| Quote: |
| If QM1 wants to send some messages to queues at QM3, it automatically creates a cluster-sender channel connecting to the cluster-receiver channel at QM3. |
If the FR is across a long, slow WAN, and the second PR is at the next port on the same switch.... It's not clear why you would want messages to take a potentially very much longer route - by way of at least another hop to the FR - from QM1 to QM3. |
Let me take an example.
Manual says that creating auto-defined channels clustering potentially relieves you from manually creating channels and managing them. But in an ideal scenario it really doesn't.
For example, i was at a client site where they had over 400 QM's in multiple clusters across the globe with about 15+ FR's. Since this was a global network, everything was behind multiple layers of firewalls and other such secure layers.
Now, if we were to propose clustering, we would have to anyway manually open up the ports for all the 400 queue managers allowing for a many to many communication. In this case, how is it different from a simple P2P setup, other than the fact that MQ created all the channels for you. However, you still end up maintaining and managing all the Open ports respective to the queue managers across all envs.
If all the traffic were flowing through FR, then firewall would be more secure or at least look good on paper where you wouldn't have to poke in so many holes.
All this is under the assumption that one has to open up some port somehow (not sure how this would be implemented but...) at the firewall level for a direct P2P setup between two Partial repositories that get auto defined channels created on demand.
KK |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Feb 04, 2008 2:22 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Quote: |
| For example, i was at a client site where they had over 400 QM's in multiple clusters across the globe with about 15+ FR's. Since this was a global network, everything was behind multiple layers of firewalls and other such secure layers. |
Well let me think a bit. The firewalls is probably the main region why they had multiple clusters.
If you now create a CLUSTER ALIAS / qmgr alias on 2 gateways you can move messages from one cluster to the other....
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Feb 04, 2008 2:33 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Your question is answered - why do cluster create fully connected networks?
If you want a solution, for a particular situation...
At a minimum, that's a different thread.
Going back to a higher level of discussion - Clustering provides two benefits: a) simplified system administration, b) workload balancing.
If your situation doesn't need those benefits, or if your situation doesn't allow those benefits...
Then don't use clustering.
Or mix clustering with P2P.
Clustering is not the hammer to all the world's nails.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| kkumar_70 |
| Posted: Mon Feb 04, 2008 2:44 pm Post subject: |
|
|
Novice
Joined: 04 Feb 2008
Posts: 10
|
| jefflowrey wrote: |
Going back to a higher level of discussion - Clustering provides two benefits: a) simplified system administration, b) workload balancing.
|
And that is the very reason there was clustering at the client site.
Again, the example that i gave was just to extend the conversation in trying to understand why would Clustering in MQ be desinged to establish a P2P over a gateway FR flow. I could state many more examples on those lines. But the point here is to reason out why Clustering in MQ is designed to P2P if it is merely doing a DQM/p2p setup under the hoods and in most cases not really relieving of manual management as was the case in this client configuration.
Jeff, i know you work for IBM as was i at one point and i would love to defend MQ to any extent possible. And believe me, that is exactly what i do in front of the customers because i love this product as much. This discussion however, is just to understand the basis for such a design and NOT to question anybody or any product's features.
KK |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Feb 04, 2008 2:55 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| kkumar_70 wrote: |
But the point here is to reason out why Clustering in MQ is designed to P2P if it is merely doing a DQM/p2p setup under the hoods and in most cases not really relieving of manual management as was the case in this client configuration.
|
In most cases MQ clustering DOES "relieve manual management".
| kkumar_70 wrote: |
| Now, if we were to propose clustering, we would have to anyway manually open up the ports for all the 400 queue managers allowing for a many to many communication. In this case, how is it different from a simple P2P setup, other than the fact that MQ created all the channels for you. However, you still end up maintaining and managing all the Open ports respective to the queue managers across all envs. |
MQ Clustering never said anything about relieving opening ports. General consensus here is your cluster shouldn't span firewalls. Put a cluster on each side and connect them to each other (P2P) with Highly Available Gateways on each side.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
|
|
