| Author |
Message
|
| bigdavem |
 Posted: Wed Mar 20, 2002 4:47 pm Post subject: |
 |
|
Acolyte
Joined: 16 Sep 2001
Posts: 69
Location: Sydney, Australia
|
I have been asked to recommend changes to our existing MQ architecture to improve reliability. We're currently running 6 qmgrs on a central hub (single NT server), and another 6 spread across NT and various flavours of unix. We have had 26 MQ-related production outages in the last year, which is unacceptably high.
The options I'm looking at are clustering, migrating to a single Win2K server (more reliable than NT and low cost to migrate ie same hardware) or migrating to a single HP-UX or AIX server.
According to an analysis of our problem logs, in a clustered environment we would have had 18 outages, on Win2K between 20 and 26 and on unix between 20 and 24. The number of outages remains high primarily because we are running a couple of platforms with v2 MQ - these would be resolved by an upgrade to v5, which we plan to do regardless of any other actions.
Clustering provides the greatest guaranteed reduction in our outages, but my manager is concerned about the complexities of clustering ie considering it may only save us a couple of outages a year, is it really worth the trouble? My question is: is clustering really so painful? What do other people do to guarantee high availability of MQ? Should we be going down the clustering path, or are we better off putting up with a couple more outages a year? |
|
| Back to top |
|
 |
| NickB |
| Posted: Thu Mar 21, 2002 2:09 am Post subject: |
|
|
Centurion
Joined: 20 May 2001
Posts: 107
Location: Zurich Financial Services
|
If you are fairly certain that you would only save a couple of outages if you went down the clustering route, then my advice would be to NOT use clustering.
Clustering is a great technology if:
- you want to simplify a complex highly distributed MQ network
- you want a form of load-balancing
- you want high availability via adding more boxes/qmgrs in to the cluster
But, you have to be committed to the technology which means you must understand exactly how it works and how to fix it when things go wrong. In a properly configured network it works well and shouldn't need any administration at all.
At the end of the day, the call is yours. Use it by all means but get yourself trained and really experienced in how it works prior to rolling it out. |
|
| Back to top |
|
|
| mrlinux |
| Posted: Thu Mar 21, 2002 6:31 am Post subject: |
|
|
Grand Master
Joined: 14 Feb 2002
Posts: 1261
Location: Detroit,MI USA
|
Well I have been working with v5.x for almost 4 years and I dont think I can
come close to 26 outages in the 4 years. Well if you are using v2 I would upgrade asap to v5.2.
Just curious what are some of the reason for your outages, you might get more mileage out
resolving those than going to Clustering
_________________
Jeff
IBM Certified Developer MQSeries
IBM Certified Specialist MQSeries
[ This Message was edited by: mrlinux on 2002-03-21 06:33 ] |
|
| Back to top |
|
|
| bigdavem |
| Posted: Thu Mar 21, 2002 2:45 pm Post subject: |
|
|
Acolyte
Joined: 16 Sep 2001
Posts: 69
Location: Sydney, Australia
|
On the plus side, the actual cutover to a cluster would be performed by our outsource partner, who have quite good MQ skills, so that part of it doesn't worry me so much.
A breakdown of the 26 outages:
4 were due to problems with a v2 client which hangs from time to time. We can't upgrade to v5 as it's on an exotic platform, but we believe we have developed a workaround, so these should be resolved regardless of any other action.
2 were network related, isolating the queue managers. Not really MQ's fault and not much we can do about it from the MQ side of things.
4 were config errors or required maintenance.
9 were OS failures.
4 were MQ channel problems, primarily to do with the v2 queue manager I mentioned. Again, it's an exotic platform. There's no v5.2 server available. There is v5, and we'd like to upgrade, but we would first need to upgrade the OS and the application which runs on the machine.
3 were queue manager failures.
|
|
| Back to top |
|
|
| bduncan |
| Posted: Fri Mar 22, 2002 2:03 pm Post subject: |
|
|
Padawan
Joined: 11 Apr 2001
Posts: 1554
Location: Silicon Valley
|
Why do you have 6 queue managers on the central server, as oppossed to 1? In my mind, you could best leverage clustering by spending the money to purchase 6 separate machines, each hosting 1 of the 6 queue managers. Assuming all 6 queue managers serve the same function, put all 6 machines in a cluster, this way, messages bound for the central server can go to any one of the 6 queue managers, and workload balancing will allow your system to continue running even if up to 5 of the 6 queue managers go down....
_________________
Brandon Duncan
IBM Certified MQSeries Specialist
MQSeries.net forum moderator |
|
| Back to top |
|
|
| bigdavem |
| Posted: Sun Mar 24, 2002 9:44 pm Post subject: |
|
|
Acolyte
Joined: 16 Sep 2001
Posts: 69
Location: Sydney, Australia
|
| The 6 queue managers aren't the same - different queue managers are used to provide logical isolation of different functions. That's the only reason I can come up with, anyway. So there's not much point spreading them across 6 boxes.... |
|
| Back to top |
|
|
| khoban |
| Posted: Tue Mar 26, 2002 8:30 am Post subject: |
|
|
Novice
Joined: 25 Mar 2002
Posts: 10
|
Taking the point of six machines a step further....
For the logical separation of business functions, could you not just use clustered aliases to non-clustered queues? This would facilitate six machines _and_ allow logical separation, but give visibility to all participants.
For example, QM1 has a queue for customer query, and defines a local queue CUSTOMER.QUERY.LOCAL.QUEUE. This queue is NOT clustered. An alias is then defined CUSTOMER.QUEUE.ALIAS.QUEUE with a TARGQ of CUSTOMER.QUERY.LOCAL.QUEUE. This alias is part of the cluster.
I had to do something similar to this for a client...
Hope this helps.
Kieran |
|
| Back to top |
|
|
| bigdavem |
| Posted: Tue Mar 26, 2002 2:32 pm Post subject: |
|
|
Acolyte
Joined: 16 Sep 2001
Posts: 69
Location: Sydney, Australia
|
I'm having trouble convincing management that clustering is the way to go. They're concerned about the complexity and would prefer just to migrate to a more stable platform than NT. I'm not too excited by that idea because it won't provide anywhere near the same robustness as a cluster - sure, it may reduce the number of OS-related outages, but it will have no impact on the outages due to MQ itself. I'm just trying to get a feel as to whether or not clustering is as painful as management believe.
I think telling them that I'd actually like to cluster 6 machines instead of the original 2 may not go down too well.... |
|
| Back to top |
|
|
| bduncan |
| Posted: Wed Mar 27, 2002 3:08 pm Post subject: |
|
|
Padawan
Joined: 11 Apr 2001
Posts: 1554
Location: Silicon Valley
|
"I'm just trying to get a feel as to whether or not clustering is as painful as management believe"
Where did they get this impression anyway? Have any of them had experience with MQ clustering before?
_________________
Brandon Duncan
IBM Certified MQSeries Specialist
MQSeries.net forum moderator |
|
| Back to top |
|
|
| mrlinux |
| Posted: Thu Mar 28, 2002 4:30 am Post subject: |
|
|
Grand Master
Joined: 14 Feb 2002
Posts: 1261
Location: Detroit,MI USA
|
Well mgmt was a little concerned when I wanted to go to clustering
I setup 4 linux box's and showed them how easy it was to make it work, plus
on how easy we could add more broker boxes as the load required with very little
effort, They let me go ahead and implement it, in fact after the first phase worked so well they wanted to speed up the implementaion.
I had this setup working in about 3 days, which included loading linux/MQSeries
our Brokers/Application (compiling) plus the rest of my duties.
1) One Box was the SOURCE Application
2) Two Boxs were for our Message Brokers Clustered Queues lived here .
3) One box for the Destination Application.
_________________
Jeff
IBM Certified Developer MQSeries
IBM Certified Specialist MQSeries
[ This Message was edited by: mrlinux on 2002-03-28 04:30 ]
[ This Message was edited by: mrlinux on 2002-03-28 04:31 ] |
|
| Back to top |
|
|
| bigdavem |
| Posted: Mon Apr 01, 2002 10:18 pm Post subject: |
|
|
Acolyte
Joined: 16 Sep 2001
Posts: 69
Location: Sydney, Australia
|
| No, we haven't done any MQ clustering here, so management have no scary stories to go on. The reason they've suddenly become worried is because one of the big wigs from our parent company was out here a few weeks ago and when I mentioned that we were planning to cluster he said something along the lines of "clustering is always more trouble than it's worth". I don't think he'd even had experience with MQ clustering, just with OS clustering. |
|
| Back to top |
|
|
| bduncan |
| Posted: Tue Apr 02, 2002 3:21 pm Post subject: |
|
|
Padawan
Joined: 11 Apr 2001
Posts: 1554
Location: Silicon Valley
|
You might want to keep this in mind. I've been told from friends at IBM that the developers at Hursley are putting two things above all else for version 5.3: security and clustering. Apparently huge strides are going to be made in both of these areas in 5.3, especially regarding the reliability of clusters...
_________________
Brandon Duncan
IBM Certified MQSeries Specialist
MQSeries.net forum moderator |
|
| Back to top |
|
|
| khoban |
| Posted: Wed Apr 03, 2002 3:57 am Post subject: |
|
|
Novice
Joined: 25 Mar 2002
Posts: 10
|
Had heard the same thing about 5.3. There were issues around placing extra integrity around the cluster channels and securing them as both of these have been raised as customer requirements and have also been challenged at user group meetings.
But this depends on whether you can migrate to 5.3...
Kieran |
|
| Back to top |
|
|
|
|
