| Author |
Message
|
| visasimbu |
 Posted: Thu Jun 14, 2012 9:53 am Post subject: Performance of the message transfering between queue manager |
 |
|
Disciple
Joined: 06 Nov 2009
Posts: 171
|
Hi
We are checking the performance of our MQ and MB setup. The setup is looking as below.
we have Q1 and Q2 are in clusters. These two queue managers are connected with the G1 queue manager by the two different sender channels (i.e not by cluster with G1).
Here Q1 and Q2 are sending messages 150 messages per second by the remote(Q1,Q2), transmission(Q1,Q2) and local queue (G1).
Due to some reason message are slowly send it to the local queue of G1 when we put billions of messages from the remote queue(Q1 and Q2). Messages are piled up in the transmission queue and slowly send it to the local queue.
Could you suggest to improve the spead of this action. |
|
| Back to top |
|
 |
| Vitor |
| Posted: Thu Jun 14, 2012 10:07 am Post subject: Re: Performance of the message transfering between queue man |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| visasimbu wrote: |
| Could you suggest to improve the spead of this action. |
Improve the speed of the network.
Improve the reliability of the network.
Ensure the channels are properly tuned for the load.
Etc.
Etc.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| visasimbu |
| Posted: Thu Jun 14, 2012 10:17 am Post subject: |
|
|
Disciple
Joined: 06 Nov 2009
Posts: 171
|
Thanks for the reply..
Could you please suggest any property of channel / elaborate below suggestion.
| Quote: |
Ensure the channels are properly tuned for the load.
|
Like to update one more thing here.. Q1 and Q2 queue manger are reside in the windows and G1 reside on the AIX platform. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Jun 14, 2012 10:28 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Ensure that servers at both ends are provisioned with sufficient cpu power and real memory (RAM).
Ensure that the physical channels (NIC cards and physical media) are not shared with other workloads.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Jun 14, 2012 10:31 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| visasimbu wrote: |
| Could you please suggest any property of channel / elaborate below suggestion. |
Message batch size is an obvious one. The use of channel exits and encryption is another.
But what you're describing sounds a lot like a slow and/or unreliable connection.
| visasimbu wrote: |
| Like to update one more thing here.. Q1 and Q2 queue manger are reside in the windows and G1 reside on the AIX platform. |
Well anything running on Windows is inherently slower than any other platform.... 
But the problem doesn't seem to be your platform (150 mps is adequate if not impressive). The problem seems to be the speed with which they're moving off the Windows platform onto the AIX, which brings us back to the network.
There could be something in the Windows network configuration that's slowing it down. I'll let others with more platform knowledge speak to that.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Jun 14, 2012 7:54 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Vitor,
Please also remember that nothing will help if at the destination the application cannot scale and everything is backing up to the channel and the xmit queue...
In this case everything is fine but the consuming application... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jun 15, 2012 4:56 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| fjb_saper wrote: |
Vitor,
Please also remember that nothing will help if at the destination the application cannot scale and everything is backing up to the channel and the xmit queue...
In this case everything is fine but the consuming application... |
Umph......
You'd hope that you'd get better than 300 mps from an AIX application, or if you couldn't you'd define a local queue deep enough to hold the predicted backlog so the slow moving reading appplication could chug through them.
You also hope the OP would have been given a clue by the "Queue Full" alerts being thrown by the AIX box.
I accept completely both these hopes may be futile & optimistic & the scenario you describe is plausible.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Jun 15, 2012 5:16 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| fjb_saper wrote: |
Vitor,
Please also remember that nothing will help if at the destination the application cannot scale and everything is backing up to the channel and the xmit queue...
In this case everything is fine but the consuming application... |
Message consumer app behavior at the receiving end of the channel has naught to do with messages stacking up in the smit queue.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jun 15, 2012 5:41 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| bruce2359 wrote: |
| fjb_saper wrote: |
Vitor,
Please also remember that nothing will help if at the destination the application cannot scale and everything is backing up to the channel and the xmit queue...
In this case everything is fine but the consuming application... |
Message consumer app behavior at the receiving end of the channel has naught to do with messages stacking up in the smit queue. |
If the consuming app is too slow to keep up with traffic and the receiving queue is too shallow to contain the backlog and the receiving queue manager has no dead letter queue then messages will pile up in the xmitq.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| visasimbu |
| Posted: Fri Jun 15, 2012 5:52 am Post subject: |
|
|
Disciple
Joined: 06 Nov 2009
Posts: 171
|
Thanks for your replies.
In my scenorio, I have stopped the receiving application to pick up the messages from the local queue of G1. But my local queue in G1 has enough space to accomadate billions of messages. Though the transmission queue is piled up with more messages and it is updating/sending messages to the local queue very slowly. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Jun 15, 2012 5:56 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Vitor wrote: |
If the consuming app is too slow to keep up with traffic and the receiving queue is too shallow to contain the backlog and the receiving queue manager has no dead letter queue then messages will pile up in the xmitq. |
A slow consuming app does not cause the xmit queue to fill up.
But, I'll accept your premise if, in fact, the logs document that the channel fails, and goes into retry - which the OP did not report.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Jun 15, 2012 5:58 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
It is not yet clear from what has been written where the "slowness" is happening.
Regardless, the original poster should be looking at the channel involved, and not anything else. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jun 15, 2012 8:33 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| visasimbu wrote: |
| But my local queue in G1 has enough space to accomadate billions of messages. |
Post details of your maxdepth, average message size and a df -k of the file system in question to back up that proud boast.
| visasimbu wrote: |
| Though the transmission queue is piled up with more messages and it is updating/sending messages to the local queue very slowly. |
Which brings us straight back to my original point regarding network and/or channel configuration.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jun 15, 2012 8:34 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqjeff wrote: |
| Regardless, the original poster should be looking at the channel involved, and not anything else. |
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Jun 15, 2012 9:21 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Vitor wrote: |
| mqjeff wrote: |
| Regardless, the original poster should be looking at the channel involved, and not anything else. |
|
Well the channel may move messages at a very slow pace if at the other end the MCA goes into retry mode everytime it tries to put a message to the queue. Ultimately the messages may end up in the DLQ on the target qmgr or the channel may stop entirely (no DLQ), but in the meantime the delay due to the retry of the destination MCA can and will cause a backup on the XMITQ or SCTQ...
Everything to do with channel settings (because you can configure this behavior of the receiving MCA), but nothing to do with network and normal operations...
Ultimately and if the consuming application is just as fast as the channels MCA retry, you may never see a message in the DLQ and the channel just seems to have slow throughput. The destination queue, although full or nearly full gets consumed... just not fast enough...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
