| Author |
Message
|
| rxm8778 |
 Posted: Wed Jan 23, 2008 7:29 am Post subject: Message size impact on MQ cluster traffic |
 |
|
Apprentice
Joined: 15 Sep 2005
Posts: 37
|
I have been pondering  upon the following question:
How does message size affect the overall latency of message traffic in an MQ cluster?
Knowing that all messages departing from a queue manager regardless of its destination in the cluster use the same xmit queue system.cluster.transmit.queue, will messages of a larger size (20MB+) slow down the overall flow of smaller messages (<100KB) leaving that same queue manager? |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Wed Jan 23, 2008 7:32 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
It will only "slow down" messages going to the same qmgr.
Oddly enough, the people who wrote MQ clustering considered this problem when doing the design.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| rxm8778 |
| Posted: Wed Jan 23, 2008 8:02 am Post subject: |
|
|
Apprentice
Joined: 15 Sep 2005
Posts: 37
|
Thanks Jeff, I was leaning toward that answer myself but did not want to go with my gutt feeling alone.
With that said if messages going to the same queue manager end up slowing down, one option would be to create a direct sender/receiver channel to that queue manager to isolate that larger traffic. But that would eliminate our use of clustering for the application sending those larger messages...
I am just curious to know if there any other options? |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Jan 23, 2008 8:11 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Nothing prevents you from creating more than one clusrcvr.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| rxm8778 |
| Posted: Wed Jan 23, 2008 9:15 am Post subject: |
|
|
Apprentice
Joined: 15 Sep 2005
Posts: 37
|
Hmmm that's true....So.... if I understand you correctly...
If I have two queue managers A and B belonging to the same cluster and I define two cluster receivers on queue manager B;
when queue manager A needs to send messages to queue manager B, queue manager A will dynamically create two cluster sender channels to B. That should compensate for any slowdow caused by a mix of small and "larger" messages heading to the same queue manager since I now have 2 channels moving messages instead of 1 but unlike the option I stated in my previous post it will not isolate the message with larger size and have it stick on a lane of its own. correct?
And that would be fine, I just want to make sure I correctly interpreted your suggestion (which sounds great since it is still using clustering  ). |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Jan 23, 2008 9:36 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
No. You create 2 clusters "stacked" on top of each other. The small messages are using queues clustered to CLUSTERSMALL, and you have cluster channels clustered to CLUSTERSMALL.
You then cluster the queues for the big messages to CLUSTERBIG, and create another set of Cluster channels clustered to CLUSTERBIG.
Use Namelists to have the Full Repositories be FRs for both clusters, and to cluster objects to both clusters equally (like QM Alias definitions).
We've done this for years.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Jan 23, 2008 9:38 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| rxm8778 wrote: |
Hmmm that's true....So.... if I understand you correctly...
If I have two queue managers A and B belonging to the same cluster and I define two cluster receivers on queue manager B;
when queue manager A needs to send messages to queue manager B, queue manager A will dynamically create two cluster sender channels to B. That should compensate for any slowdow caused by a mix of small and "larger" messages heading to the same queue manager since I now have 2 channels moving messages instead of 1 but unlike the option I stated in my previous post it will not isolate the message with larger size and have it stick on a lane of its own. correct?
And that would be fine, I just want to make sure I correctly interpreted your suggestion (which sounds great since it is still using clustering ). |
Yes.
It's not two overlapped clusters.
It's two channels, in the same cluster, that will get load balanced individually - since clustering always and only works via the channel.
So half the messages will still be stuck behind the big one.
But more messages should go to the other channel, while the big one is stuck. MAYBE.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| rxm8778 |
| Posted: Wed Jan 23, 2008 10:33 am Post subject: |
|
|
Apprentice
Joined: 15 Sep 2005
Posts: 37
|
Gotcha... thanks for the insight...
Some actual performance test exercises from my part will help clear out that "MAYBE" but you put me on the right track...
thanks again. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Jan 23, 2008 10:38 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Use stacked clusters and there won't be a maybe. The smaller messages will not be impacted by the bigger ones because each will be serviced exclusivly by their dedicated channel.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Jan 23, 2008 10:41 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Assuming you can control when and where apps send BIG messages... 
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Jan 23, 2008 10:56 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
If you cluster the queues for the big messages to CLUSTERBIG, and then grant the big applications authority to only be able to put to their queues you should be fine.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Jan 23, 2008 10:58 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| PeterPotkay wrote: |
| If you cluster the queues for the big messages to CLUSTERBIG, and then grant the big applications authority to only be able to put to their queues you should be fine. |
Assuming one of the SMALL apps doesn't decide to send a BIG message.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Jan 23, 2008 11:08 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Yup. A problem not specific to clustering. Never understood why IBM didn't include Max Message Length on Remote Queues and Alias Queues. 
If the apps are MQ Clients you could control this via the channel parms.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| rxm8778 |
| Posted: Wed Jan 23, 2008 12:24 pm Post subject: |
|
|
Apprentice
Joined: 15 Sep 2005
Posts: 37
|
| PeterPotkay wrote: |
| Use stacked clusters and there won't be a maybe. The smaller messages will not be impacted by the bigger ones because each will be serviced exclusivly by their dedicated channel. |
Over here most if not all our queue managers are part of a cluster to begin with. My only problem with stacked cluster would be that each time a new project wants to send larger messages to a queue manager, and we have concerns that it might affects remaining smaller message traffic to that same queue manager, we would have to create/stack a cluster on top of the larger cluster just for that new traffic. Wouldn't this overall increase the administrative and mainteance work that cluster is meant to reduce?
| jefflowrey wrote: |
| Nothing prevents you from creating more than one clusrcvr. |
Actually I just gave this additional cluster receiver channel option some more thought and I got a little skeptical. Here is why: usually regardless of the amount of queue managers that are in a cluster, only one cluster receiver channel is defined on each queue manager. Yet, every other queue manager in that cluster can create a dynamic cluster sender channel to a destination queue manager QMB in the same cluster (creating more than one inbound channel into QMB). So how does creating an additional cluster receiver channel on QMB affect the amount of channel created only between QMB and QMA. Hope this makes sense...I am almost lost trying to explain it myself...  |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Jan 23, 2008 12:27 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Creating an additional clusrcvr means that every qmgr can(will) create an additional clussdr to the qmgr. This includes a new channel from QMA to QMB.
On the flip side, nothing says you can't create an additional (manually defined) clussdr from QMA to QMB, so that you only have additional throughput on that one leg, rather than everywhere.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
