| Author |
Message
|
| Gideon |
 Posted: Thu Jul 01, 2010 12:03 pm Post subject: Express queues writing to disk |
 |
|
Chevalier
Joined: 18 Aug 2009
Posts: 403
|
I am using WMQ 7.1 on Linux
I am writing data to an express queue, and am getting disk activity when large volumes of data are being sent.
I need to eliminate all disk IO, and have all data contained inside a buffer.
I defined the qm.ini with the following:
| Code: |
#*
TuningParameters:
DefaultQBufferSize=104857600
DefaultPQBufferSize=104857600
Log:
LogPrimaryFiles=16
LogSecondaryFiles=2 |
However, I can see disk activity when I write to the queue. Also, top is reporting that I have 30 gig free (I have a total of 32 gig), I have 82 queues defined after defining the DefaultQBufferSize to the above value
The 82 queues should have taken about 16 gig from my free space, so I should see about 14 gig free.
Why am I not able to convince WMQ to use more heap space when defining the queues.
I changed the qm.ini, then restarted, then created the queues, and then restarted again for good measure.
Any ideas ? |
|
| Back to top |
|
 |
| Gideon |
| Posted: Thu Jul 01, 2010 12:05 pm Post subject: |
|
|
Chevalier
Joined: 18 Aug 2009
Posts: 403
|
BTW, when I display a QL(queue) command, is there a variable that will show me how much heap will be allocated for the express buffer ?
Thanks |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Jul 01, 2010 12:30 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
What is an "express queue"?
Are you using persistent or non persistent messages?
Are you using the syncpoint option on the puts?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Gideon |
| Posted: Thu Jul 01, 2010 12:42 pm Post subject: |
|
|
Chevalier
Joined: 18 Aug 2009
Posts: 403
|
Sorry about the 'express queue' phrase
I am using non-persistent messaged on a stadard local WMQ queue.
We are not using a syncpoint option on the put |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Jul 01, 2010 1:02 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Non-persistent messages UNDER syncpoint can be passed directly to the in-memory buffer of an application that has an open MQGet waiting to get them.
All other messages MUST go to the q file.
I realize you are attempting to control the writes to the q file by setting a very large memory buffer for those disk writes. There could be a number of reasons why this is not happening, however.
You should review the performance supportPacs and do some digging on the IBM site for redbooks and etc that may apply.
Then you may need to open a PMR to get a better idea of what's going on and whether what you are trying to do is possible or not. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Jul 01, 2010 1:25 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| mqjeff wrote: |
Non-persistent messages UNDER syncpoint can be passed directly to the in-memory buffer of an application that has an open MQGet waiting to get them.
|
But the syncpoint processing will need to write to the logs for the check points, so that would show disk activity, even if the messages themselves may be passing via memory.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Gideon |
| Posted: Thu Jul 01, 2010 3:14 pm Post subject: |
|
|
Chevalier
Joined: 18 Aug 2009
Posts: 403
|
I was previously having the clients send non-persistent messages to an AIX using WMQ7.1. I used a qm.ini file on that qmgr with the following parameters:
| Code: |
#*
TuningParameters:
DefaultQBufferSize=104857600
DefaultPQBufferSize=104857600
Log:
LogPrimaryFiles=16
LogSecondaryFiles=2 |
That worked well, and I never had any disk writes or disk activity while populating the quuees.
Now I have moved the queues to a Linux machine (Wmq7.1), and am using the exact same qm.ini parameters, but when the same client application sends the non-persistent message, I get disk activity.
I deleted the queues, and stopped the qmgr. Then I restarted the qmgr and created them again. Then I stopped and started for good measure. I did all of this one more time.
This is all that I have read in the docs and support pacs to do. Hoever I still get the disk activity.
Are there any other areas or things I could try. Any suggestions or hints ?
Thanks |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Jul 01, 2010 6:11 pm Post subject: Re: Express queues writing to disk |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| Gideon wrote: |
| I need to eliminate all disk IO, and have all data contained inside a buffer. |
Why? Security? Performance?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| exerk |
| Posted: Thu Jul 01, 2010 10:23 pm Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
How have you determined that the disk IO is WMQ-related? Have you considered any difference in spec between the Linux and AIX servers (physical) and OS?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| Gideon |
| Posted: Fri Jul 02, 2010 5:55 am Post subject: Re: Express queues writing to disk |
|
|
Chevalier
Joined: 18 Aug 2009
Posts: 403
|
| PeterPotkay wrote: |
Why? Security? Performance? |
Performance |
|
| Back to top |
|
|
| Gideon |
| Posted: Fri Jul 02, 2010 5:57 am Post subject: |
|
|
Chevalier
Joined: 18 Aug 2009
Posts: 403
|
| exerk wrote: |
| How have you determined that the disk IO is WMQ-related? Have you considered any difference in spec between the Linux and AIX servers (physical) and OS? |
I am still trying to determine if this is exactly WMQ related. Do you have any suggestions on how to do this ?
Thanks |
|
| Back to top |
|
|
| exerk |
| Posted: Fri Jul 02, 2010 5:58 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
Your *nix admins may have monitoring tools that identify the processes making calls on resources.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Jul 02, 2010 6:10 am Post subject: Re: Express queues writing to disk |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| Gideon wrote: |
| PeterPotkay wrote: |
Why? Security? Performance? |
Performance |
You've identified exactly how long your MQPUTs and MQGETs are taking and they are too slow? Or you are just trying to squeeze every last bit of performance out of the design, even though the MQ part may be fast enough?
Is MQ the worst offender in your design? No sense in you going nuts trying to shave a millisecond of the MQ part when some other part like a database transaction is wasting seconds.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Gideon |
| Posted: Fri Jul 02, 2010 9:24 am Post subject: |
|
|
Chevalier
Joined: 18 Aug 2009
Posts: 403
|
I think I found the problem.
The WMQ was routed to WMB, which had 5 executino groups. filled with flows
I deleted the execution groups and repopulated them with the exact same flows
Then the disk activity stopped altogether.
The flows did not contain DB or file access, just a simple XSLT transformation.
So I do not know why this would stop disk activity. Has anyone seen this behavior from WMB ? |
|
| Back to top |
|
|
|
|
