| Author |
Message
|
| csongebalazs |
 Posted: Tue Apr 02, 2013 6:08 am Post subject: very slow MQ 7.5 IO on windows 2008 |
 |
|
Voyager
Joined: 30 Jan 2004
Posts: 78
|
Hi,
An MQ 7.5 server was installed on a windows 2008 enterprise R2 64 bit server.
With a simple message flow i generated test messages for a performance test and this message generation was very slow.
On a win 2003 server with MQ7 the generation of 60K messages finished less then 1.5 minute, and same process on win2008 - MQ7.5 lasts more then 15 mins.
Clearing messages from the destination queue was same slow (with using MQGET API calls).
Watching the resource usage during these actions (on win2008 with resource monitor) i saw 0 CPU, 0 memory usage, but disk IO usage of amqzmuch0.exe was at 100%. The write/read speed was only around 300K/sec which seems too low.
I made a file copy, which done with 28M/sec.
Win2003 server has a single disk without any raid, and this win2008 has 2 disks in raid. Can it be the main source of the problem?
Why can simple file copy done so fast, but MQ IO not?
Regards,
Balázs |
|
| Back to top |
|
 |
| soaretudor |
| Posted: Wed Apr 03, 2013 1:22 am Post subject: |
|
|
Newbie
Joined: 03 Apr 2013
Posts: 4
|
Hello,
Are there any FDC files created by this process?
Normally if there is an error with this process we should at least have some FDC files with some indepth info.
Thanks |
|
| Back to top |
|
|
| csongebalazs |
| Posted: Wed Apr 03, 2013 1:56 am Post subject: |
|
|
Voyager
Joined: 30 Jan 2004
Posts: 78
|
There was not any error, simply putting or getting 60K messages to/from a queue was very slow.
Made same operations on my win7 laptop with MQ7.5 and WMB8 and was not any performance problem.
We disabled disk mirror, but it not helped.
I had to realize our support guys presented us a developer server from the garbage. It's some SUN server from year 2007, without SCSI controller. Most drivers from 2006. It contains 2 ST1000dm003-9yn162 disks.
Anyway, after checking in the "Turn off Windows write-cache buffer flushing on the device" check-box under Device-Manager\Disk drivers\xy Disk Device\Policies\, the performance increased significantly.
Just a remark:
On the other win 2003 server, which performed well, during this tests, in the same parameter path, the "Enable advanced performance" check-box is not checked. |
|
| Back to top |
|
|
| Esa |
| Posted: Wed Apr 03, 2013 4:07 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
Are you generating persistent or nonpersistent messages?
How many messages do you write within the batch?
Are you reading messages from the queue while you are writing new ones? |
|
| Back to top |
|
|
| csongebalazs |
| Posted: Wed Apr 03, 2013 4:47 am Post subject: |
|
|
Voyager
Joined: 30 Jan 2004
Posts: 78
|
In that point of view, when i doing SAME thing on 2 different server and it perform very different speed, persistence and others must not be important.
I know putting persisntent messages more resource hungry... All message persistent.
I put 10K, 20K and 30K messages to 3 different queues.
No, not putting and getting in same time.
Getting was a very simple right click on queue and clear messages with api calls. |
|
| Back to top |
|
|
| Esa |
| Posted: Wed Apr 03, 2013 5:03 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| csongebalazs wrote: |
Getting was a very simple right click on queue and clear messages with api calls. |
You did not answer my question about the number of messages. I mean, getting from a queue that has a large number of large messages just is slow. I guess in your case there wasn't much difference between MQ 7.5 and 7? |
|
| Back to top |
|
|
| csongebalazs |
| Posted: Thu Apr 04, 2013 2:02 am Post subject: |
|
|
Voyager
Joined: 30 Jan 2004
Posts: 78
|
I think I gave the exact number of messages. The test message generator flow, whcih has 1 compute node, generates 10000 + 20000 + 30000 persistent messages into 3 different queue. There are 6 do-while loops which do this job, and all loops put 10000 message to its target queue.
The test message generation flow totally same on win2003+MQ7+MB7, on win2008+MQ7.5+MB8 and on win7+MQ7.5+MB8, and was slow only on win2008.
Based on this, MQ and MB version seems not the source of the problem, only win2008 and/or the actual machine could be that.
And because getting of messages with MQ explorer was as same slow as message generation, the test generating flow doesn't important.
I think the garbage server can be the main source, which cannot support this IO hungry operations without that windows settings. But not 100% sure, so maybe has a chance, the windows 2008 file management is different from win2003 and from win7? |
|
| Back to top |
|
|
| Esa |
| Posted: Thu Apr 04, 2013 6:32 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| csongebalazs wrote: |
Anyway, after checking in the "Turn off Windows write-cache buffer flushing on the device" check-box under Device-Manager\Disk drivers\xy Disk Device\Policies\, the performance increased significantly. |
I wonder if doing this in fact also decreases the reliablility level from assured persistent to reliable persistent, if we talk in JMS terms... it depends on if the Windows write cache is on disk or in-memory.
I think you may be right, Windows 2008 file system seems be the cause.
| csongebalazs wrote: |
I think I gave the exact number of messages.'
|
Oh, I thought 10K+ 20K +30K were messages sizes. So it's the message sizes then that you haven't told.
| csongebalazs wrote: |
And because getting of messages with MQ explorer was as same slow as message generation, the test generating flow doesn't important.
|
I'm sorry but I cannot comprendre/capire/megert what you are trying to say. |
|
| Back to top |
|
|
| csongebalazs |
| Posted: Thu Apr 04, 2013 11:58 pm Post subject: |
|
|
Voyager
Joined: 30 Jan 2004
Posts: 78
|
Message data size does not exceed 1000 chars in case of any message.
Sorry for my English. I wanted to say that: first time I was not sure, the test message generator flow or something else is the source of the slowness. But after I realized, a simple way of geting messages (with MQ explorer clear messages from the queue using MQGET API calls) is same slow as putting with a flow, the message flow and/or the message broker efficiency couldn't be the reason. |
|
| Back to top |
|
|
|
|
