| Author |
Message
|
| fde |
 Posted: Thu Feb 18, 2010 1:06 am Post subject: Performance issues MQI Channel vs Message Channel |
 |
|
Acolyte
Joined: 05 Jul 2007
Posts: 65
|
I encountered very different performance results on those two channel types.
In my test, I put 100 messages with a size of 4 MB on a remote queuemanager.
First, using mqputsc with a MQI channel.
Second, putting on a remote queue on a local quemanager, which sends the messages through a Message channel to the remote queuemanager.
The results do not change if the test messages are already on the transmission queue and the channel gets started to issue the test.
In the first test, all messages reached the remote queuemanager within 180 seconds.
In the second test, all messages reached the remote queuemanager within 280 seconds.
Why could the message channel be slower than the mqi channel?
Are there properties for the channels, which could influence my results?
MQ Version is 6.0
_________________
Global warming is an unintentional side effect of SOA's hotness.
-- http://soafacts.com/
a business integration methodology |
|
| Back to top |
|
 |
| zpat |
| Posted: Thu Feb 18, 2010 1:54 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Because the sender channel sends a batch of messages between turnaround points.
Try using a WMQ v7 client channel (presumably need a WMQ v7 queue manager) you should see an improvement (not sure if you have to specify any option to make the client channel using batching). |
|
| Back to top |
|
|
| fde |
| Posted: Thu Feb 18, 2010 3:44 am Post subject: |
|
|
Acolyte
Joined: 05 Jul 2007
Posts: 65
|
Thank you for the input.
The sender channel has a batchsize of 50 and a batch intervall of 0.
As long as the connection dont break frequently during 50 messages, this value seems ok.
With those values, could the sender channel perform slower considering my test?
_________________
Global warming is an unintentional side effect of SOA's hotness.
-- http://soafacts.com/
a business integration methodology |
|
| Back to top |
|
|
| mvic |
| Posted: Thu Feb 18, 2010 4:28 am Post subject: Re: Performance issues MQI Channel vs Message Channel |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| fde wrote: |
In the first test, all messages reached the remote queuemanager within 180 seconds.
In the second test, all messages reached the remote queuemanager within 280 seconds. |
In your second test you have 100x4Mb=400Mb more data written to disk, and 100x4Mb=400Mb more data read from disk, on the sending side. I am a little surprised this gives 100 seconds more elapsed time, though.
Are your messages persistent? Are they being put inside syncpoints? Is your disk holding the xmitq fast? |
|
| Back to top |
|
|
| fde |
| Posted: Thu Feb 18, 2010 4:56 am Post subject: |
|
|
Acolyte
Joined: 05 Jul 2007
Posts: 65
|
Good point with the creation of the messages, since mqputsc performs this task very fast.
The messages are persistent and the queuemanager should have sufficient fast disk access.
I attached the channeldefinitions of the transmission channels:
| Code: |
CHANNEL(DUX151D03/HMM05) CHLTYPE(RCVR)
ALTDATE(2010-01-13) ALTTIME(08.32.01)
BATCHSZ(50) COMPHDR(NONE)
COMPMSG(NONE)
DESCR(SDR chl. security with coding AES)
HBINT(300) KAINT(AUTO)
MAXMSGL(4194304) MCAUSER( )
MONCHL(QMGR) MRDATA( )
MREXIT( ) MRRTY(10)
MRTMR(1000) MSGDATA( )
MSGEXIT( ) NPMSPEED(FAST)
PUTAUT(DEF) RCVDATA( )
RCVEXIT( ) SCYDATA( )
SCYEXIT( ) SENDDATA( )
SENDEXIT( ) SEQWRAP(999999999)
SSLCAUTH(REQUIRED)
SSLCIPH(TLS_RSA_WITH_AES_128_CBC_SHA)
SSLPEER(CN=dux151d03)
STATCHL(QMGR) TRPTYPE(TCP)
CHANNEL(HMM05/DUX151D03) CHLTYPE(SDR)
ALTDATE(2010-01-18) ALTTIME(15.22.17)
BATCHHB(0) BATCHINT(0)
BATCHSZ(50) COMPHDR(NONE)
COMPMSG(NONE)
CONNAME(dusu0151.xyz.net(13103))
CONVERT(NO)
DESCR(SDR chl. security with coding AES)
DISCINT(6000) HBINT(300)
KAINT(AUTO) LOCLADDR( )
LONGRTY(999999999) LONGTMR(1200)
MAXMSGL(4194304) MCANAME( )
MCATYPE(PROCESS) MCAUSER( )
MODENAME( ) MONCHL(QMGR)
MSGDATA( ) MSGEXIT( )
NPMSPEED(FAST) PASSWORD( )
RCVDATA( ) RCVEXIT( )
SCYDATA( ) SCYEXIT( )
SENDDATA( ) SENDEXIT( )
SEQWRAP(999999999) SHORTRTY(10)
SHORTTMR(60)
SSLCIPH(TLS_RSA_WITH_AES_128_CBC_SHA)
SSLPEER(CN=dux151d03)
STATCHL(QMGR) TPNAME( )
TRPTYPE(TCP) USERID( )
XMITQ(DUX151D03_XMITQ)
|
_________________
Global warming is an unintentional side effect of SOA's hotness.
-- http://soafacts.com/
a business integration methodology |
|
| Back to top |
|
|
| mvic |
| Posted: Thu Feb 18, 2010 5:49 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
So as far as you have told us, there is still the possibility that persistent messages are being put outside syncpoints (this is an extremely important point to establish with your application people by the way), and there is the certainty that you have at least 800 Mb more disk I/O in the slower case. I am not sure but it could be 1200 Mb, because there will be write/read on the xmitq q file, and write to log files.
Message channels and MQI channels are very different in purpose. Please can you give some details about why you are measuring relative performance of the two?
Last edited by mvic on Thu Feb 18, 2010 7:11 am; edited 2 times in total |
|
| Back to top |
|
|
| fde |
| Posted: Thu Feb 18, 2010 7:00 am Post subject: |
|
|
Acolyte
Joined: 05 Jul 2007
Posts: 65
|
The reason I compare those two channels is, that I had general performance issues between the two queuemanagers, despite a 4GBit connection between them.
Some basic tests using mqputs from ih03 performed not as good as expected. The tests taken afterwards showed very different behavior for those two channeltypes.
While searching for performance improvements, I tried to understand the different behavior of the channels.
I understand your IO argument and I think too, that this shouldnt have such a great impact, if IO is working in normal terms.
I am currently not able to provide the synchpoint configuration for the queuemanager but I try to post it tommorow.
_________________
Global warming is an unintentional side effect of SOA's hotness.
-- http://soafacts.com/
a business integration methodology |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Feb 18, 2010 7:04 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
In my test, I put 100 messages with a size of 4 MB on a remote queuemanager.
First, using mqputsc with a MQI channel: 180 seconds.
Second, putting on a remote queue on a local quemanager, which sends the messages through a Message channel to the remote queuemanager: 280 seconds. |
A bit surprising; but there are likely other issues to consider. One test does not a benchmark make.
Same network used for both tests?
Qmgrs configured similarly (same hardware, same software, workloads)?
Do the same tests with messages of smaller and larger sizes. Do the same tests with more and less messages. See if you get different or similar results.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| mvic |
| Posted: Thu Feb 18, 2010 7:17 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| fde wrote: |
| I understand your IO argument and I think too, that this shouldnt have such a great impact, if IO is working in normal terms. |
Let's quantify it. How long does it take on your system to write 800 Mb and read 400 Mb from disk? My guess is, over 20 seconds if the disk is not already loaded. If puts are outside syncpoint then the effect could be magnified.
| Quote: |
| I am currently not able to provide the synchpoint configuration for the queuemanager but I try to post it tommorow. |
The information will need to be found from either tracing your putter application while it is running, or inspecting the putter application source code. It is not part of your qmgr's configuration. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Feb 18, 2010 9:57 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
And nobody here has yet spoken about residual load on the queue manager and how this figures in the log file # and size considerations.
It could possibly be that the log files are just a little bit too small for the coverage he needs? This could potentially have a serious impact on performance...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| fde |
| Posted: Fri Feb 19, 2010 7:52 am Post subject: |
|
|
Acolyte
Joined: 05 Jul 2007
Posts: 65
|
To seperate network issues, I did some tests in the lan of the target queuemanager.
So instead of a remote queuemanager sending messages through the WAN. A queuemanager in the local lan, sends messages to the queuemanager.
Issuing 100 x 4MB, the messages were put onto the first queuemanager within 10seconds. All messages arrived at the target queuemanager within roghly 13 seconds (3s issuing selects on queuedepth until all messages arrived).
This seem to be sufficient fast as one could expected. Since the target queuemanger is the same from the first test, I assume the network connection to have a decent impact on the performance.
The definition of the channels between the two queuemanagers is posted above. Could there be some misconfiguration or missing customiziation which leads to slow transfers?
Pinging the queuemanger machine through the wan using maximum sized ping packets stated 25 ms latency.
_________________
Global warming is an unintentional side effect of SOA's hotness.
-- http://soafacts.com/
a business integration methodology |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Feb 19, 2010 8:21 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Did you rerun your original tests? You might have had a transient problem.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| fde |
| Posted: Mon Feb 22, 2010 1:01 am Post subject: |
|
|
Acolyte
Joined: 05 Jul 2007
Posts: 65
|
I rerun the test through the wan. It is as slow as in the first try. Roughly 180 seconds for 100x 4 MB using mqputs from a windows client.
_________________
Global warming is an unintentional side effect of SOA's hotness.
-- http://soafacts.com/
a business integration methodology |
|
| Back to top |
|
|
| mvic |
| Posted: Mon Feb 22, 2010 3:36 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
No surprises about the network then:
- run over a slow network, takes a long time to transfer messages
- run over a fast network, takes a short time to transfer messages
Do you still have the big difference between client channel and message channel? Did you find the syncpoint options in use by the apps? |
|
| Back to top |
|
|
| fde |
| Posted: Mon Feb 22, 2010 4:28 am Post subject: |
|
|
Acolyte
Joined: 05 Jul 2007
Posts: 65
|
About the network. It should be very fast. The fastest wan connection avaiable as far as i know with a bandwith of 4 GBit.
Concerning the synchpoints. Since I am using mqputs from ih03 as my client. I dont exactly know about the implementation of commiting. I guess that it is optimized for performance rather interoperability and uses one commit statement at the end of the operation.
_________________
Global warming is an unintentional side effect of SOA's hotness.
-- http://soafacts.com/
a business integration methodology |
|
| Back to top |
|
|
|
|
