| Author |
Message
|
| joper |
 Posted: Mon May 09, 2011 11:02 pm Post subject: MQ Performance |
 |
|
Newbie
Joined: 09 May 2011
Posts: 3
|
Hi,
I have 2 applications which are using MQ.
1st app ---put---> MQ Queue1 <---get--- 2nd app ---put--> MQ Queues
All are running on the same machine.
The 1st app can send 2500msg/sec but the 2nd app only has 1300msg/sec. We tested the 2nd app to only send to a file but its performance is so high. So we assume that the bottleneck is in the MQ. Any idea how to increase the performance of MQ? Normally, the 2nd app will send the msg to 15-20 MQ queues.
Thanks! |
|
| Back to top |
|
 |
| Mr Butcher |
| Posted: Mon May 09, 2011 11:13 pm Post subject: |
|
|
Padawan
Joined: 23 May 2005
Posts: 1716
|
for every plattform (IIRC) there is a performance white paper on the supportpacs site. you should verify your application settings and environment with that to get a feeling of what performance you can expect. just because app1 is faster than app2 does not make mq a bottleneck. To me, your application look quite fast. but there are many factors that have an impact on the performance, e.g. message persistence, running with syncpoints or without, ......
unfortunately, there is no turbo stanza in the qm.ini file ..
_________________
Regards, Butcher |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue May 10, 2011 2:04 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
| Mr Butcher wrote: |
| unfortunately, there is no turbo stanza in the qm.ini file .. |
Well there kind of is (fastpath), but then I would not direct anybody to use it, unless directed by IBM to do so. If your code has any flaws, it could kill your qmgr really fast... and in these days of "try it first and fix later", it is best to stay away from.... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| zpat |
| Posted: Tue May 10, 2011 2:37 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5849
Location: UK
|
You could try using log SingleWrite instead of TripleWrite, on modern SAN this should be quite safe.
You could increase the log buffer size (2048 worked well for me).
You could try increasing the PQBufferSize, but you will need to delete and re-create the queue to make this take effect. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue May 10, 2011 6:12 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9396
Location: US: west coast, almost. Otherwise, enroute.
|
Before assuming that it is MQ that is allowing the producing application to produce messages pretty quickly, but preventing the consuming app from keeping up, I'd suggest looking at how the slower consuming app is (badly) coded.
For example:
Is the consuming app a client-bindings app? Does it MQCONNect and MQDISConnect for each MQGET?
Does the producing app create large UofWs? Thus making the consuming app wait for the producer to MQCMIT?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| joper |
| Posted: Tue May 24, 2011 8:21 pm Post subject: |
|
|
Newbie
Joined: 09 May 2011
Posts: 3
|
Sorry for the late reply. We just tested today the performance of our app. The consuming app has only 1500msg/s which too far from our target(2500msg/s).
Our current configuration settings are
LogPrimaryFiles=7
LogSecondaryFiles=5
LogFilePages=4096
LogType=CIRCULAR
LogBufferPages=4096
LogPath=/var/mqm/log/MyQueueManager/
LogWriteIntegrity=TripleWrite
The only change we did was increasing the LogBufferPages to 4096. So there is not much change from the performance last time with LogBufferPages set to default. We also cannot risk to set the integrity to SingleWrite.
| bruce2359 wrote: |
Before assuming that it is MQ that is allowing the producing application to produce messages pretty quickly, but preventing the consuming app from keeping up, I'd suggest looking at how the slower consuming app is (badly) coded.
For example:
Is the consuming app a client-bindings app? Does it MQCONNect and MQDISConnect for each MQGET?
Does the producing app create large UofWs? Thus making the consuming app wait for the producer to MQCMIT? |
The consuming app only do connect and disconnect once the app is started, not every MQGet. What it actually does is get as much as possible messages that is available in the MQ Queue. It does not wait for the producer app to MQCMIT. |
|
| Back to top |
|
|
| joper |
| Posted: Tue May 24, 2011 10:06 pm Post subject: |
|
|
Newbie
Joined: 09 May 2011
Posts: 3
|
| one more thing is the cpu usage for the consuming app is very low, only 3%. I wonder why when it is very busy getting and sending message to mq queues. |
|
| Back to top |
|
|
| zpat |
| Posted: Tue May 24, 2011 10:13 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5849
Location: UK
|
Increase the unit of work size (do more puts and gets before the commit).
SingleWrite is safe to use, providing your disk technology ensures writes are written even if the power fails.
Can you multi-thread any of this?
For persistent messages, the speed looks good. Check out the IBM performance reports and see how yours compares. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed May 25, 2011 8:58 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
Also you did not specify if the gets were ALL done within syncpoint.
You should seriously consider running multiple instances/threads of the consuming application. AFAIK the MQGET is always somewhat slower than the MQPUT. The difference there is mostly negligible except maybe in cases of extreme high volume....
Looking at your description I would look at the application. Having to put to 15 to 20 queues will take some time. You will not be able to run as fast as an application putting to a single queue.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed May 25, 2011 9:36 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9396
Location: US: west coast, almost. Otherwise, enroute.
|
| fjb_saper wrote: |
| AFAIK the MQGET is always somewhat slower than the MQPUT. |
How so?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu May 26, 2011 2:54 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
| bruce2359 wrote: |
| fjb_saper wrote: |
| AFAIK the MQGET is always somewhat slower than the MQPUT. |
How so? |
Note that I also said that the difference was negligible except in case of extreme volume?
Well reading the message in java and discarding it, I was never able to achieve the same rates as putting messages... It also greatly depends on how big your unit of works are and typically the get units of work are smaller than the put units of work. (i.e. you may put 1,000 msgs in a single unit of work, but we would not read the same 1,000 msgs in a single unit of work. Especially if each message represents a single atomic transaction. You'd want to minimize your rollback size...)
It also depends on the connection type used by the putting and getting application (bindings vs client)
It may also depend on message size. Although I did not make any test with large messages as to the put time, I have noticed that large messages (3.5 MB vs 10 kB for instance) take significantly more time in the get/receive than small ones.
And finally you will probably also see some difference according to programming language (java vs cobol zos, vs C) and platform. These differences should also be quite negligible... but may matter in cases of extreme volume...
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
