| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Problems with circular logging & checkpoint question |
« View previous topic :: View next topic » |
| Author |
Message
|
| cecils |
 Posted: Thu Mar 13, 2003 7:05 pm Post subject: Problems with circular logging & checkpoint question |
 |
|
Newbie
Joined: 11 Jun 2002
Posts: 7
|
This concerns MQSeries V.5.2
Hi everyone. We are currently experiencing some problems with out MQ Series configuration (running on Win2K Advance Server). When we configured the queue manager before, we just accepted all the default values for circular logging (256 pages, 3 primary and 2 secondary log files) since we didn't expect big messages to be sent to MQ. We usually expect messages with size ranging from 1-44 kb. We also set the queue to persist messages. We didn't really encounter any problems before especially during our load testing when we are sending 18000+ messages (sizes range from 1-2kb). But right now, there came a requirement to send big messages to MQ with size ranging from 500kb to 2.5 mb. We are now encountering a MQRC_RESOURCE_PROBLEM and on the event log, its saying that the log file of the queue manager is full (that is after sending 2 2.5 mb messages). We tried increasing the number of log files (both primary and secondary) but the same error occurred after sending about 21 messages with 2.5 mb sizes. Then I've read about checkpoints and that MQ will only release the log files after a checkpoint. And checkpoints usually occurs after 1000 transactions.
Here are my questions:
1. Can we set how often should MQ do a checkpoint?
2. We want to avoid doing this but if there's no choice, we need to redefine the size of each of the log file (we have 2 GB physical space allocated for MQ alone). What setting would you recommend considering the size that will be coming into our MQSeries server (2.5 mb to 4 mb per message). Volume per day is quite high since this concerns an internet transaction.
Thank you very much in advance. |
|
| Back to top |
|
 |
| jhalstead |
| Posted: Fri Mar 14, 2003 1:15 am Post subject: |
|
|
Master
Joined: 16 Aug 2001
Posts: 258
Location: London
|
First off as of version 5.2 I believe checkpoints are taken every 10,000 operations (Get, Put Commit etc). I understand that if a queue manager realises it is about to run out of log space it will force a checkpoint to be taken.
You cannot configure this checkpointing interval - or if you can I've not heard about it..
In your current configuration 256 pages = 1Mb per log file.
This gives a maximum total log file volume of 63Mb (63 is tha max no of primary and secondary.) Now if you are pumping thousands of messages of some size on to this tiny log then I'd fully expect resource type errors.
So, the queue manager would like to maintain the last checkpoint in it's circle of logs so there is no way that this is going to work with the current configuration!
Now as you've got 2Gb to play with I don't think we need to worry.
Lets assume that the typical life cycle of a mesage on this qmgr is as follows.
1) Msg Put
2) Msg Commit
3) Msg Get
4) msg Commit
So 4 operations are performed for every message.
10,000 / 4 = 2500.
So take your average message size, lets go with 100Kb.
2,5000 * 100Kb = 250Mb
Add 20% for MQ operations under the covers that requie sapce also.
So 300Mb / 63 gives us a log file size of ~ 5Mb = 1280 pages.
I'd go with 42 Primaries and 21 secondaries. The primaries will always exist and secondaries will only be created if they are needed.
Now there is a good chance that even if there is a wave of large messages (exceeding the total available log space) the queue manager will have enough time to recognise this and force a checkpoint before any resource limits stop your applications...
Hope this is some help
Jamie |
|
| Back to top |
|
|
| beu |
| Posted: Fri Mar 14, 2003 1:58 am Post subject: |
|
|
Newbie
Joined: 12 Mar 2003
Posts: 4
Location: Frankfurt, Germany
|
Hello
increasing the log file is ok, but IBM suggest having less log files with a bigger size instead of many with a tiny size. this propably will take more performance on mqseries.
Windows max page count per log file is 4095 (file size 16774144 bytes)
so for about 300Mb log you'll need only 20 primary log files
 beu |
|
| Back to top |
|
|
| pgorak |
| Posted: Fri Mar 14, 2003 2:20 am Post subject: |
|
|
Disciple
Joined: 15 Jul 2002
Posts: 158
Location: Cracow, Poland
|
Pasted from V5.2 documentation:
Checkpoints are generated automatically by MQSeries. They are taken when the queue manager starts, at shutdown, when logging space is running low, and after every 1000 operations logged.
Piotr |
|
| Back to top |
|
|
| jhalstead |
| Posted: Fri Mar 14, 2003 3:33 am Post subject: |
|
|
Master
Joined: 16 Aug 2001
Posts: 258
Location: London
|
Good point Beu, indeed this would be better as you can increase the number of log files without a re-build - whereas you need to re-build to increase log file size!
W.R.T operations per checkpoint:
Pasted from v5.2 documentation.
"Third edition (August 2001)
This edition applies to the following products:
MQSeries for AIX(R) Version 5.2
MQSeries for HP-UX Version 5.2
MQSeries for Linux Version 5.2
MQSeries for Sun Solaris Version 5.2
MQSeries for Windows NT(R) and Windows(R) 2000 Versions 5.2 and 5.2.1 "
"Checkpoints are generated automatically by MQSeries. They are taken when the queue manager starts, at shutdown, when logging space is running low, and after every 10 000 operations logged. "
Hell, whats an order of magnitude difference anyway?! |
|
| Back to top |
|
|
| sknrt1 |
| Posted: Fri Mar 14, 2003 1:45 pm Post subject: |
|
|
Apprentice
Joined: 22 Jan 2003
Posts: 39
Location: USA
|
Hi,
The way to achieve more message handling is to increase the Log File Size (~4095 Pages in windows for each log file). This may cause performance effect, but it can be avoided, by dedicating the system resources for MQ (not having any other huge processes).
The Current Log File Pages = 256 (in u r system), and it needs to redefine the Queue Manager to change lof File Pages.
At present the no of log files can be set to 20 or 40 accoding to req, and it can be increased later on upto 63. The change of no of file requires, restart of QM.
The check point process is carried by MQSeries automatically.
thanks
 |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
