| Author |
Message
|
| GeneRK4 |
 Posted: Fri Jul 26, 2013 11:46 pm Post subject: Load balacing in MQ |
 |
|
Master
Joined: 08 Jul 2013
Posts: 220
|
Hi,
We have 1 Gateway Queue manager and 2 Broker Queue managers in a Cluster.
Recently we faced issues as messages are pending in SYSTEM.CLUSTER.TRANSMIT.QUEUE.This caused some slowness in the messages and hence timeout errors occured at endsystems.
In the meanwhile,for a new Application,the customer has come up with a requirement that without having cluster setup (without SYSTEM.CLUSTER.TRANSMIT.QUEUE) ,the messages should be load balanced between two broker Queue managers.
The design requirement is the external applications will connect to brokers via gateway queue manager.But the gateway queue manager should not use SYSTEM.CLUSTER.TRANSMIT.QUEUE to contact the broker Queue managers but workload balancing and high availability should be happening between the broker Queue managers.Since the SYSTEM.CLUSTER.TRANSMIT.QUEUE is the only queue used for multiple high volume realtime transanctions,they have comeup with this requirement.
Is this kind of design approchable in MQ ?
If we have an Alias queue in Gateway queue manager,again internally it uses SYSTEM.CLUSTER.TRANSMIT.QUEUE for communicating with broker Queue managers.
I am not sure when Gateway,Broker QMs are in cluster,having a Remote Queue definition in Gateway QM to contact Broker local queue would be an idea.In that case,the transmit queue will restrict the messages only for a particular broker queue manager.
Please help whether there is an approachable solution for this requirement.  |
|
| Back to top |
|
 |
| PeterPotkay |
| Posted: Sat Jul 27, 2013 3:47 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
In MQ 7.5 you can have multiple cluster transmit queues.
http://pic.dhe.ibm.com/infocenter/wmqv7/v7r5/topic/com.ibm.mq.pro.doc/q001050_.htm
So the Gateway would have a cluster XMITQ for each Broker, but that would still be shared by all other apps sending messages to that broker.
You could then take it a step further and set up an overlapping dedicated cluster with dedicated cluster transmit queues for this app.
You are solving for the symptom instead of the root problem.
What caused the original S.C.T.Q. to back up? What leads you to believe that when the problem happens again it wouldn't impact your dedicated cluster transmit queues or regular transmit queues the same way?
If this new apps messages are so important use MQMD.Priority.
If this new app is doesn't like how MQ load balances, create regular SND/RCVR channels between the gateway and the brokers, create a remote queue def for each Broker on the gateway that resolves to the XMITQs that feed these SNDR channels, and have the application code its own load balancing routine that does everything the internal MQ Cluster Workload algorithm does. Let us know how that works out.
Or, you could have the app make client connections directly to the Broker queue managers and again put the load balancing responsibilities on them.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mqjeff |
| Posted: Sat Jul 27, 2013 4:39 am Post subject: Re: Load balacing in MQ |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| GeneRK4 wrote: |
| Please help whether there is an approachable solution for this requirement. |
Get a new customer. Fire this one. Tell them they don't know what they're doing, and you won't work for them.
MQ workload balancing is performed using MQ clusters. That's it. There is no other mechanism, unless you write your own.
Do not write your own. It's a waste of time, and is a single point of failure. (unlike a single pig of failure, from a recent barbecue)
The client channel table does not do workload balancing, for those pedants in the wings. It provides connection balancing, not workload balancing. |
|
| Back to top |
|
|
| zpat |
| Posted: Sat Jul 27, 2013 4:48 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
If you have multiple app servers (and/or multiple threads on each server) accessing the same queue (on the same QM) they will load balance themselves.
This also avoids the problem of orphaned messages. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Sat Jul 27, 2013 8:12 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| zpat wrote: |
If you have multiple app servers (and/or multiple threads on each server) accessing the same queue (on the same QM) they will load balance themselves.
This also avoids the problem of orphaned messages. |
They won't load balance themselves.
They just won't ever get the same message.
Each instance of the app will get the next available message, when it is ready to process another message. This could mean that one instance of the app gets all messages.
That's not load balancing. |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Jul 30, 2013 7:14 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
If two apps servers are reading the same queue, and both process message at approx the same rate they will be load-balanced because they will be making requests for the messages at a roughly even rate and MQ will satisfy them in the order the requests were made.
Believe me - this technique is superb, robust, simple and very large organisations use it.
If one apps server goes down - the others carry on, no orphaned messages. In fact even the message currently being processed will be returned to the queue (syncpoint rollback) and another apps server will get it.
It is also great where one apps server is faster than the other since they will just take what they can handle, rather than being given messages that they might not be able to handle in a timely manner.
It's also not a SPOF if the queue manager is HA protected. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Jul 30, 2013 7:41 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| zpat wrote: |
| If two apps servers are reading the same queue, and both process message at approx the same rate they will be load-balanced because they will be making requests for the messages at a roughly even rate and MQ will satisfy them in the order the requests were made. |
Yes, but there's nothing that ensures that the apps will process messages at approximately the same rate.
And there's nothing that can alter the distribution of the load to account for differences in the rate of processing.
Again, there's no mechanism to actually balance the load.
It is at best, load sharing. Not balancing. |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Jul 30, 2013 9:09 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
You still don't get it. Messages are NOT pushed to apps.
They consume, process - repeat. So the rate of consumption is based on their own processing rate by definition.
There is no need for a load-balancing mechanism. There is NO distribution.
There are simply "n" apps helping themselves to the messages as they process them.
Perfect, simple, elegant, robust. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jul 30, 2013 9:22 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| zpat wrote: |
| Perfect, simple, elegant, robust. |
And reliant on a single queue manager being able to service the client connections and the get requests of n applications for all values of x, where n is the number of applications needed to meet the SLA and x is the maximum depth of the queue at peak message transmission time. In which case it's exactly as you describe & I've used it myself.
If you require a value of n larger than the single queue manager can manager or even with n applications x grows to a depth where SLAs are impacted, you then need to balance across 2 or more queue managers.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Jul 30, 2013 9:33 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| zpat wrote: |
| You still don't get it. |
I absolutely do get it.
 
| zpat wrote: |
Messages are NOT pushed to apps.
They consume, process - repeat. So the rate of consumption is based on their own processing rate by definition. |
I completely agree.
| zpat wrote: |
| There is no need for a load-balancing mechanism. There is NO distribution. |
I agree. That's exactly what I am saying. It's *NOT* load-balancing.
I am being very technical and very precise about what it means to BALANCE the LOAD. And what you have described DOES NOT BALANCE THE LOAD.
I'm not saying it's bad. I'm not saying it's wrong.
I'm merely saying that it's NOT load balancing. |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Jul 30, 2013 10:03 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| Such pedantry is not really helpful to most people here, who would just want to be able to have multiple servers process a common workload. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 31, 2013 4:34 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| zpat wrote: |
| Such pedantry is not really helpful to most people here, who would just want to be able to have multiple servers process a common workload. |
I thought my comments about when your solution is a great idea and when you may need to actually consider a WMQ cluster both relevant & potentially helpful to future readers considering if they should set up a cluster or not. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Jul 31, 2013 7:02 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
And if some consultant goes in and sets up your solution and then a month later, one server shows that it's processing 100% of the work and the other server is sitting entirely idle....
Is it unnecessary pedantry to explain that "Oh, yes, I know I said it would be load balanced, but I really meant it would be load shared, with no mechanism to enforce any distribution between servers"?
Please don't confuse my technical accuracy for an attack on your solution.
It's absolutely the right thing for a large number of situations.
 |
|
| Back to top |
|
|
| markt |
| Posted: Wed Jul 31, 2013 7:08 am Post subject: |
|
|
Knight
Joined: 14 May 2002
Posts: 508
|
In most cases, the "pull" model is fine, but sometimes it's not.
There are some people who absolutely desire some level of real workload balancing. Often this is for reasons that are not technical (eg pricing).
And that's why Lyn recorded this 3-part presentation about MQ workload distribution in a z/OS sysplex: http://www.youtube.com/playlist?list=PL9N7JP2yU3T8JycrCOvEPM8c-0UdE97VT |
|
| Back to top |
|
|
| zpat |
| Posted: Wed Jul 31, 2013 7:10 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
If both servers are running the MQ App, they will both process load in accordance with their own capacity.
It is not possible for one to do 100% because whilst it is busy processing one message, the other server will have a MQGET outstanding on the queue and will get the next message.
This works best in conjunction with long-running MQGETs of course.
It's much better than MQ clustering because the load is pulled and not pushed (regardless of whether anything is actually working).
Really don't see your problem here. It's much better in nearly all respects to MQ clustering for the purpose of allowing multiple application servers to process a shared workload.
Did I mention that it saves money on MQ licenses as well? |
|
| Back to top |
|
|
|
|
