| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| How to improve queue performance on WIN NT queue manager |
« View previous topic :: View next topic » |
| Author |
Message
|
| sysnemr |
 Posted: Tue Nov 26, 2002 6:56 am Post subject: How to improve queue performance on WIN NT queue manager |
 |
|
Apprentice
Joined: 29 Oct 2001
Posts: 30
Location: Zagreb, Croatia
|
How to improve queue performance and reach the maximum number
of GETs per second in all queues on NT queue manager? How does greater size than default size of log files help to improve queue performance or to increase number of GETs per second? How does defined max depth of queue influence to queue performance and to application performance? I'd like to avoid the max depth in queues because I expect in future lot of messages in queues. I don't know can my production queue managers and application process fast in that way that queues don't become full. My question is: Can I help to application to clean the queues faster with proper configuration of queues and queue manager or this problem only depends upon application design, server hardware configuration and database? The version of MQSeries for WIN NT is 5.1. |
|
| Back to top |
|
 |
| jhalstead |
| Posted: Tue Nov 26, 2002 9:53 am Post subject: |
|
|
Master
Joined: 16 Aug 2001
Posts: 258
Location: London
|
The obvious things that spring to my mind in no particular order!
The MQ to application performance is very much dependent on the approach being applied, request reply, fire and forget etc etc
1)use non persistent messages.
2)use larger log file sizes, the larger they are the less frequent they have to be created.
3)build in parrallelism into MQI applications, i.e. many processes can drain messages form the same queue.
4)avoid having to browse through many many messages as this is a linear sequential process.
5)if you have to message match then increase the size of the memory buffers
6)increase the number of memory pages allocated to the log buffer
7)seperate log and object filesystems on differnet disks
8)use hardware to further boost performance, more CPU, more memory, fast disks...
9)connector perform in a batched mode, i.e. get 10 messages before a commit, rather than get, commit, get commit etc... |
|
| Back to top |
|
|
| sysnemr |
| Posted: Wed Nov 27, 2002 5:52 am Post subject: |
|
|
Apprentice
Joined: 29 Oct 2001
Posts: 30
Location: Zagreb, Croatia
|
Long time ago I improved queue manager performance with larger
allocation of log files. So, your 2nd advice is OK. Today morning I started with testing to put lot of messages to queue. I noticed that putting lot
of messages to queues doesn't have any impact to log files. They remained the same, unchanged with same date and time. Also, second
allocation was not allocated. So, my conclusion is that log files are allocated correctly. Only queue directory was changed. Can you explain your 5th and 6th advice once again? Out NT Paging File Size is 600M. We use circular log with default values. In application we use get,commit,get,commit....... We use persistent queues and messages. We don't have separeted log and object file systems on different disks. Our application programmers would try to process messages faster. Problem is that arrival of new messages to queue with trigger is faster than process which gets the messages. Do you have any new ideas what to do here to improve queue performance? |
|
| Back to top |
|
|
| nimconsult |
| Posted: Wed Nov 27, 2002 6:40 am Post subject: |
|
|
Master
Joined: 22 May 2002
Posts: 268
Location: NIMCONSULT - Belgium
|
The recommendations of jhalstead are an excellent first inventory.
You have yourself described many of the reasons why you have performance issue: no separation of data and log files, proces one message at a time (get, commit, get, commit...).
My first question is what kind of throughput are you talking about: some messages per second or hundreds of messages per second? In the first option my comment is: if you cannot consume messages as fast as they are produced, it's not MQ Series, look at your app first!
In a "get, commit, get, commit" pattern on a single disk, producer, consumer and MQ Series running on single platform with single CPU I would say that MQ Series can process a minimum of 50 persistent messages per second.
You are talking about paging file size of 600MB. I would not take that into account. If Windows is doing pagination at a constant rate, definitly your configuration is unadapted: increase your memory, not your paging file size.
You are talking about a queue with trigger. What kind of trigger? In high throughput systems never use trigger every. Windows will die by starting processes! Either use trigger first if one instance of the server application is enough, or start a fixed number of server application instances to process messages in parallel (as proposed by jhalstead).
If your application (client and/or server) is running on the same machine, concurrent disk activity may also impact MQ Series.
CPU may also be a source of contention if you run other apps in parallel to MQ Series.
_________________
Nicolas Maréchal
Senior Architect - Partner
NIMCONSULT Software Architecture Services (Belgium)
http://www.nimconsult.be |
|
| Back to top |
|
|
| jhalstead |
| Posted: Fri Nov 29, 2002 3:39 am Post subject: |
|
|
Master
Joined: 16 Aug 2001
Posts: 258
Location: London
|
Neville re your questions:
Can you explain your 5th and 6th advice once again?
5) if message matching increase size of queue buffers...
When you perform a get you can specify match options on things like msgid & correlid. This requires a scan to be performed through all the messages on the queue. If the queue buffer is larger is size more queue information/data is in memory and the chances are you can locate your message without needing to go to disk.
This can be controlled through the tuning stanza of qm.ini (on unix platforms) defaul size 64Kb, max 1Mb.
Add:
TuningParameters:
DefaultQBufferSize=256000
TO BE HONEST I'M NOT SURE HOW YOU DO THIS ON WINDOWS... Blush...
6)increase the number of memory pages allocated to the log buffer
This should improve small message through put when using persistent messaging.
Again on unix this in controlled in the the qm.ini file. default 17, max 512.
Log:
LogFilePages=512
Again, not sure if/how this is done on windows... However I'm sure someone on the forum will know!
The questions posed by Nimconsultant will help the targetting of any further assistance. However splitting log & queue filesystems across different disks is always a favourite.
Jamie
Last edited by jhalstead on Fri Nov 29, 2002 10:59 am; edited 1 time in total |
|
| Back to top |
|
|
| jhalstead |
| Posted: Fri Nov 29, 2002 3:43 am Post subject: |
|
|
Master
Joined: 16 Aug 2001
Posts: 258
Location: London
|
Ah Ha!
HKEY_LOCAL_MACHINE\Software\IBM\MQSeries\CurrentVersion\Configuration\QueueManager\
<NameOfYourQueueManager>\TuningParameters
DefaultQBufferSize =128000
&
HKEY_LOCAL_MACHINE\Software\IBM\MQSeries\CurrentVersion\Configuration\QueueManager\
<NameOfYourQueueManager>\Log
LogBufferPages=512 |
|
| Back to top |
|
|
| nimconsult |
| Posted: Fri Nov 29, 2002 4:57 am Post subject: |
|
|
Master
Joined: 22 May 2002
Posts: 268
Location: NIMCONSULT - Belgium
|
Some more info:
If I remember well, the unofficial default queue buffer size parameter applies to the queues that you create after setting the attributes. It does not affect the existing defined queues of your system.
When you change the parameter, the queue manager must be restarted.
There is no command to see the queue buffer size attached to a given queue.
Caution must be taken with this parameter, if you set it high to every queues of your system, you will end up swapping and loosing the benefit.
The parameter is described in the "Performance Highlights" document.
_________________
Nicolas Maréchal
Senior Architect - Partner
NIMCONSULT Software Architecture Services (Belgium)
http://www.nimconsult.be |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
