| Author |
Message
|
| MB |
 Posted: Tue Aug 16, 2005 2:44 pm Post subject: Limit on Number of Clusters... |
 |
|
Acolyte
Joined: 25 Jun 2004
Posts: 52
|
Hi,
Say, we have 100 participating Queue Managers and 2 Repository Queue Managers. For business needs, I have to separate the 100 Queue Managers into 10 different clusters (couple of Queue Managers are part of all the 10 clusters).
Can the 2 Repository Queue Managers hold the repository for all the 10 clusters? Is there a limit on this number of clusters?
Ok, I have to put all these different cluster names in a namelist and specify that in the REPOSNL attribute of the QM; here I have a limitation of 256 names. So, will a Queue Manager be able to hold the repository for all these 256 clusters? Or is there any functional limit that is less than 256?
Where can I find IBM recommendations on such questions?
Is there an IBM recommendation on the max number of Queue Managers within a cluster?
Thanks and Regards,
MB |
|
| Back to top |
|
 |
| Nigelg |
| Posted: Tue Aug 16, 2005 10:46 pm Post subject: |
|
|
Grand Master
Joined: 02 Aug 2004
Posts: 1046
|
Where can I find IBM recommendations on such questions?
Is there an IBM recommendation on the max number of Queue Managers within a cluster?
| Quote: |
| Can the 2 Repository Queue Managers hold the repository for all the 10 clusters? |
YES
| Quote: |
| Is there a limit on this number of clusters? |
Yes, the number of names in a namelist.
| Quote: |
So, will a Queue Manager be able to hold the repository for all these 256 clusters? Or is there any functional limit that is less than 256?
|
These are the same questions as above, with slightly different wording.
| Quote: |
Where can I find IBM recommendations on such questions?
Is there an IBM recommendation on the max number of Queue Managers within a cluster?
|
There are no IBM recommendations. This is an implementation question, not a technical one. For an opinion on this you will have to pay for IBM consulting or sefvices.
The most qmgrs in a single cluster I know of is more than 3000. Thefe is no specific limitation on the numebr of qmgrs in a cluster.
_________________
MQSeries.net helps those who help themselves.. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Aug 17, 2005 3:22 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I would recommend solving this problem a different way. I don't know what that other way is, because I don't know the actual requirements, only what you've stated. But the business requirements for separating the queue managers likely have to do with access to resources - and clustering is a poor way to manage access control.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| hopsala |
| Posted: Wed Aug 17, 2005 3:46 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
I agree with Jeff. Clusters are complex and fragile things to maintain, and if you have any other reasons other than backup or workload balancing, there are other, better solutions at hand.
Could you post the exact buisness scenario and requirements? |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Aug 17, 2005 3:51 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| hopsala wrote: |
| Clusters are complex and fragile things to maintain, |
I don't agree with that, necessarily. In my experience, they work fine once you get them set up right, and setting them up right is well documented.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| hopsala |
| Posted: Wed Aug 17, 2005 1:37 pm Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
I've had a rather different experience with clusters. Granted usually once you have them set up in a small framework (not too many qm's) everything is ok, however, in a large network problems begin to creep up.
This is much improved since 5.3 somewhere around CSD07, but still clusters have one big problem - I know they are supposed a self-managed transparent way of handling the network, problem is, they're TOO transparent and too self managed. I have come across a few cases in which a slight error in configuration (wrong ip in clusrcvr and such) creates havoc in the cluster - msgs getting stuck, qm's being disconnected from the clusters - and the greatest problem was that the msgs in the error logs were completely incoherent and did not point to the right place.
Also, the fact remains that from time to time the cluster suddenly goes defunct (shared queues not published etc) and the only fix is to recreate the cluster; in my eyes this is a poor joke for an infrastructure.
I think that it is a wide custom of veteran MQ admin's to forget just how complex MQ is for the firsttimer; though for us XMIT queues and trigger monitors seem like the most basic things in the world, the truth is that it takes a long time (years even) to learn how to use the full mq infrastructure properly.
But the real problem is not the fact that it took us sometime 5 years ago a good week to set up the cluster and another few weeks to truely get the hang of it - we're tough, we can take it  - The thing is we're not the only people in the world, and each time a new MQ admin is born, each time a new MQ programmer is brought into this world, he has to go through the same learning inferno we had.
I have been an MQ instructor for the past 5 years, and not a bad one at that, i'd like to think ; I have given over 50 official IBM courses, probably taught over 500 people what mq is all about, and believe me - MQ is hard, really really hard. |
|
| Back to top |
|
|
| MB |
| Posted: Wed Aug 17, 2005 3:37 pm Post subject: |
|
|
Acolyte
Joined: 25 Jun 2004
Posts: 52
|
Well, I realise that we have a little complex cluster design that could have been avoided and done with a simpler design to meet our requirements. But the cluster design is already in place (for more than a year) and all these discussions are just for better thoughts if any.
This is how the scenario is:
1 HQ (say Level-1 office)
10 Regional Head Offices (Level-2 offices)
100 Local Offices (Level-3 offices - 10 each under the Level-2 offices)
3 BIG applications that cater to all the offices (Since these are hosted in HQ, let me call them as Level-1 Applications)
Similarly,
2 big applications at Level-2 offices
1 big application at Level-1 offcies
All applications communicate via a Broker.
There is one Queue Manager at each big application and at the Broker.
Different programs at each big application connect to its respective QM and put messages to different queues. The local queues are shared in their respective clusters for the Hub QM to be able to put messages to them.
There are communications between two or more applications in several combinations (like Level-3 to Level-1, Level-2 to Level-1, Level-1 to Level-3 etc - but all are via the Broker)
Now, this is how the clusters are defined:
At each Level-2, the 2 Level-2 QMs and the corresponding 10 Level-3 QMs are in one cluster; this makes 10 clusters;
The 3 Level-1 QMs are in one cluster.
The Broker QM is part of all the 11 clusters. Its queues are shared in appropriate clusters (like some queues visible to only Level-3 apps and some visible to only Level-1 apps etc).
There are two Repository Queue Managers, holding repository for all the 11 clusters.
Hope I've put it clearly.
There is no workload balancing that we considered/ thought of so far.
Logical grouping of QMs is only thing I see that we achieved with this design.
We could have had just 1 cluster and share the required queues in the cluster.
Does anyone have any thoughts?
Thanks and Regards,
MB |
|
| Back to top |
|
|
| Nigelg |
| Posted: Thu Aug 18, 2005 2:06 am Post subject: |
|
|
Grand Master
Joined: 02 Aug 2004
Posts: 1046
|
I do have some thougts.
What do you gain by setting up all these clusters? The two main advantages of clustering are automatic:
1. interlinked communication between qmgrs
2. Workload balancing
You do not need 1, because the structure is hierarchical, not a flat group.
You do not need 2, because the msgs only have single destinations, either 'up' to the HQ, or 'down' to the local offices.
It would be simpler just to use normal distributed queueing to transfer msgs from the local offices through the regional offices to HQ, and back again. The cluster does not buy you anything except setup complexity for no benefit.
Even if you do use clustering, there is no point in having 11 clusters. A single cluster is sufficient, and would eliminate the need to set up the distributed channels between the HQ <-> Regional <-> Local, and not introduce any complexity except the manual channels to the full repos qmgrs. Now you are gaining something, by not having to define channels from all the local qmgrs to the regions.
To reply to hopsala...
Clusters do require careful setup. I do not know why you would think that an error in configuration, however 'slight', could not cause problems.
It is not the case that clusters suddenly go defunct. This is always caused by errors in administration or operation.
I do agree that the error msgs could be improved.
_________________
MQSeries.net helps those who help themselves.. |
|
| Back to top |
|
|
| hopsala |
| Posted: Thu Aug 18, 2005 5:30 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| Quote: |
| I do not know why you would think that an error in configuration, however 'slight', could not cause problems. |
Naturally a configuration error causes problems, my point was that in clusters an error in one server creates errors in other servers, and that the error msgs do not give any hint as to who is the qm that is the source of the prob. Any self-respecting product should definitely be able to handle simple configuration errors with better grace 
| Quote: |
| It is not the case that clusters suddenly go defunct. This is always caused by errors in administration or operation. |
We shall have to agree that we disagree on this point. My experience proved otherwise - a step-by-step setup of clusters sometimes results in a defunct cluster.
Anyway, onwards with the topic at hand... |
|
| Back to top |
|
|
| MB |
| Posted: Thu Aug 18, 2005 12:00 pm Post subject: |
|
|
Acolyte
Joined: 25 Jun 2004
Posts: 52
|
Ok, so I am in line with Nigelg (we could have done with one cluster).
And coming to the initial issue of this thread, I infer that there is no problem in adding more and more cluster names for which the Repositoy QMs have to hold the repository (I am sure it would not touch 256 in our case, it will be 15 at the most;)
Thanks a lot for the inputs.
Hmm... Clustering!!! I feel it is not normal; in the sense it definitely needs that "Extra" caution to be exercised than what is needed with implementing other stuffs. Of course, the cluster implementation steps are documented very clearly but it is human to make error/ miss something. The punishment I face for a mistake with clusters is often more severe than it is with mistake in other things. Very fact that there is a discussion and difference in opinion among several people about clustering makes it 'special'. To me, MQ Clusters are very fragile. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Aug 18, 2005 5:04 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Ask someone trying to learn how to ride a bike if he thinks that thing with only two wheels is balanced fragile like.
Computers don't make mistakes.......
BUT, unless there is a definite need for clustering, I say stay away. Why ride the bike (clustering) when you can get a ride on the bus (regular distributed MQ).
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| hopsala |
| Posted: Fri Aug 19, 2005 1:29 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| Quote: |
| Computers don't make mistakes....... |
True, but people who write software do. IBM is not omnipotent, and from time to time they write something which is so-so, see ICS for a good example of a really bad product of IBM.
I am not saying MQ is bad, not at all - it is one of the best products I know in the IT industry in general - but it is imperfect, like everything in life and software. I have been working with MQ since 1999, and have created many a cluster (though not in the past year, I admit) - and I am saddened to say that sometimes things go wrong due to no fault of mine, the cluster simply 'breaks down' and has to be rebuilt. This does not happen a lot, but it happens.
Similar reports I got from at least 5 other veteran MQ admins I know from different sites, so I do not believe we are all simply bad admins who do not admit their own mistakes... |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Aug 19, 2005 3:25 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Getting back to the original question..
There is one advantage of the multiple cluster environment that has been set up - it is an explicit model of the business. It makes the divisions of the company and where stuff is and moves around in those divisions visible and obvious.
But I'm not sure that that visibility gain outweighs the costs of the complexity added.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
