| Author |
Message
|
| sebastia |
 Posted: Fri Feb 07, 2014 12:40 am Post subject: clustering algorythm does not include queue depth |
 |
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Hi
I am reading "The cluster workload management algorithm" in mq v7.5
>>> http://pic.dhe.ibm.com/infocenter/wmqv7/v7r5/topic/com.ibm.mq.ref.con.doc/q082390_.htm
... and I am a bit surprised - the algorithm does not take into account the DEPTH of the destination queue.
Am I right ?
In our scenario, we have few clients sending messages to a cluster queue, lets call in DATAIN.
It shall be usual that DATAIN is the input to MB flow, right ?
If one flow gets into trouble, and does not cosume its input messages, we do NOT want mq cluster to still use this instance for workload balancing.
Is there any way to reduce the load assigned to this queue ?
Change its priority dynamicallky maybe ?
I know algorythm detects when a queue is not "put enabled" but to manage this from a MB flow seems not adequate to me ...
Or maybe it is ?
Thanks. Sebastian. |
|
| Back to top |
|
 |
| sebastia |
| Posted: Fri Feb 07, 2014 12:56 am Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
|
| Back to top |
|
|
| McueMart |
| Posted: Fri Feb 07, 2014 1:01 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
I agree it would be really nice if cluster workload balancing could take into account queue depth. As far as I know this capability doesnt come out the box.
I think the current only way to do it would be to write your own cluster workload exit and implement it yourself (You would need some kind of data store which constantly updated the depths of all the cluster queues - Im not sure how feasible this is!). |
|
| Back to top |
|
|
| sebastia |
| Posted: Fri Feb 07, 2014 1:22 am Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Yes, the "Omegamon agent" does just that ...
Thanks ! |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Feb 07, 2014 5:11 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7717
|
Imagine the performance implications if a queue manager had to stop and inquire on the depth of every potential destination queue in its list of queues to round robin to before deciding which queue to send to.
Also, a queue depth > 0 doesn't mean there is a problem. How could MQ know the consuming application (the flow) is "in trouble", versus it just being a normal backlog of messages due to a temporary burst of volume?
Take a look at the AMQSCLM program that now ships with MQ.
http://pic.dhe.ibm.com/infocenter/wmqv7/v7r1/topic/com.ibm.mq.doc/fg17980_.htm
| Quote: |
This sample uses the built-in WebSphere® MQ cluster workload balancing features to direct messages to instances of queues that have consuming applications attached. This automatic direction prevents the build-up of messages on an instance of a cluster queue to which no consuming application is attached.
|
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Feb 07, 2014 6:09 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
|
| Back to top |
|
|
| JosephGramig |
| Posted: Fri Feb 07, 2014 7:15 am Post subject: |
|
|
Grand Master
Joined: 09 Feb 2006
Posts: 1231
Location: Gold Coast of Florida, USA
|
Very interesting...
On the topic "inhibit PUT" on the cluster queue, in my experience I've tried this and you will have a few messages go to the DLQ between the time you inhibit the queue and the other Qmgrs learn of this and take that into account for load balancing.
Of course, this only happens under load. It would be better to just remove it from the cluster and put it back in when you want it included.
I always change the SYSTEM.DEFAULT.(alias, local, remote).QUEUE to DEFBIND(NOTFIXED). If an applications needs to bind on open, then it should say so on the dang open call. I hate that the default is bind on open. Very irritating. |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Fri Feb 07, 2014 8:10 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2602
Location: The Netherlands (Amsterdam)
|
Interesting indeed! Hadn't look at it that way...
CLWLPRTY seems to be key in this Cluster Workload Exit Sample
the default value of 0 does not work very well, when one of the Clustered Queues is in trouble as you can not lower it to -1 ...
a solution without Exit using would be to put the default of all your clustered queues at least on 1 and then you can lower the one that is in trouble to 0
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| sebastia |
| Posted: Sat Feb 08, 2014 5:02 am Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Peter - I know that "dynamic" values are difficult to include in the algorithm. You have to recalculate on real time basis.
But the algorithm does include CLWLPRTY and I hope it does "propagate" a new value I shall set in the destination queue when filling up.
Thanks for the pointer, Peter and Saper - AMQSCLM says "The cluster queue monitoring sample program uses the IPPROCS (open for input process count) value of a queue to determine whether that queue has any consumers"
It is good to resend the messages when IPPROCS is 0.
But I want to handle the situation when my flow becomes busy and queue starts growing.
Anyway, I shall look at all the programming details, as how to change CLWLPRTY - thanks a lot you both.
Joseph - that same default value makes me lose hours every time I come into cluster customers installations, jejeje.
Agree 100% to change it !
Michael - yes, CLWLPRTY is the door to solution, I'd say, if it propagates fast enough. And you are right : starting value of 0 is not good.
Have a nice weekend ! Sebastian. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Feb 08, 2014 6:48 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
Well there is some dynamic routing you can do depending on % across different machines (to take into account differences in horse power...). But the ultimate solution is to make your broker fast enough (additional instances of the flow, same flow in additional egs) and to make your queue deep enough to be able to even out peak loads... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| sebastia |
| Posted: Sat Feb 08, 2014 8:07 am Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Yes, mr Saper, I agree completely - servers must be able to eat all the messages they are provided, and we try to configure them this way.
Sometimes, we get some extra load that is difficult to manage for a while.
At those specials moments, maybe we want to release the pression from the sources, by lowering the priority of this cluster queue.
It looks good to me, by now.
Thanks a lot all of you. Sebastian. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sat Feb 08, 2014 10:44 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7717
|
You could alter the AMQSCLM program to take action based on Queue Depth versus IPROCS count.
You say this is a WMB flow that consumes from the queues you are considering this for. What do you have Additional Instances configured to for this flow?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| sebastia |
| Posted: Sun Feb 09, 2014 1:58 am Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Additional Instances ...
a) consumes machine resources
b) is also a "static" setting
Of course we shall use AI.
But want to try changing priority.
Have to see how fast the info (priority change) propagates thru the cluster ...
Cheers |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sun Feb 09, 2014 7:54 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7717
|
| sebastia wrote: |
Additional Instances ...
a) consumes machine resources
|
Only when needed. It sounds like its needed.
| sebastia wrote: |
b) is also a "static" setting
|
The broker will spin up and shut down the additional threads as needed.
http://pic.dhe.ibm.com/infocenter/wmbhelp/v8r0m0/topic/com.ibm.etools.mft.doc/ac09055_.htm
If you haven't explored Additional Instances to solve this problem I would do that first before messing around with the way MQ distributes messages.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mqjeff |
| Posted: Sun Feb 09, 2014 10:28 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| sebastia wrote: |
Additional Instances ...
b) is also a "static" setting |
in v9, it's part of the workload management infrastructure. So at a minimum, you can set up a number of workload management policies, each at different qualities of service, and apply them as needed.
You can also, of course, write your own instrumentation to alter or change the policy or the AI on demand. |
|
| Back to top |
|
|
|
|
