| Author |
Message
|
| scott9 |
 Posted: Tue Apr 26, 2005 8:24 am Post subject: Space avail.;xaction rolled back;secondary logs not created |
 |
|
Acolyte
Joined: 11 Jul 2002
Posts: 62
Location: Sacramento,CA
|
mq 5.3 csd5 on HPUX.
LOGS: pri=4, sec=58, LogType=CIRCULAR, logFilePages=4096.
Trying to send 900MB file to remote queue. 50 logs are created total. 1GB free space available in log filesystem at start of transaction. Drops to 390MB after 195 messages reach xmitq. No more secondary logs are created. Transaction rolls back after the 50th log is created. Doesn't matter if channel is active or stopped (transaction still rolls back).
Each log consumes 16MB, so I have enough space for more logs. I'm expecting more secondary logs to be created with my free space (i.e. 58+4=62, but only 50 logs are created). Any ideas?? |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Tue Apr 26, 2005 6:03 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Check max file size for your log.
Your max transaction size is 63 * your max log file size.
In this case (58+4 ) * 4096 = 253 952 or 248 MB .. a far cry from the 900 MB you are trying to put there !!!
Anyway having a transaction size of 900 MB is not a very good design. And remember if your message is persistent or your channel speed is "normal" you will need the log space on both sides of the channel.
You should look into defining better what the need is and whether or not there is a different way of satisfying it.
Enjoy   |
|
| Back to top |
|
|
| scott9 |
| Posted: Wed Apr 27, 2005 11:06 am Post subject: |
|
|
Acolyte
Joined: 11 Jul 2002
Posts: 62
Location: Sacramento,CA
|
Thanks for the info; however, I need some clarification on the math. Since the logfilesize is in 4k pages, wouldn't the algorithm look like below:
max transaction size = (#lp + #ls) * ($logfilesize * 4096)
example: lp=4, ls=58, lf=4096
( 4 + 58 ) * (4096 * 4096) = 1040187392 (~990 MB).
Just checking...I can't seem to find where this algorithm is documented. The admin and planning guides leave me wanting more information. Can anyone point me to a good reference link? Thanks in advance. |
|
| Back to top |
|
|
| Anirud |
| Posted: Wed Apr 27, 2005 11:54 am Post subject: |
|
|
Master
Joined: 12 Feb 2004
Posts: 285
Location: Vermont
|
scott9,
From System Admin Guide...
| Quote: |
| For WebSphere MQ for UNIX systems, the default number of log file pages is 1024, giving a log file size of 4 MB. The minimum number of log file pages is 64 and the maximum is 16 384. |
I think you are correct. You have allocated 992 MB of space for your log files. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Apr 27, 2005 7:23 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| scott9 wrote: |
Thanks for the info; however, I need some clarification on the math. Since the logfilesize is in 4k pages, wouldn't the algorithm look like below:
max transaction size = (#lp + #ls) * ($logfilesize * 4096)
example: lp=4, ls=58, lf=4096
( 4 + 58 ) * (4096 * 4096) = 1040187392 (~990 MB).
Just checking...I can't seem to find where this algorithm is documented. The admin and planning guides leave me wanting more information. Can anyone point me to a good reference link? Thanks in advance. |
You're right my mistake. That's what you get when you work late.... 
Anyway what are you really trying to do:?
How many messages are you putting to the queue?
How big are the messages ?
How many uncommitted messages on average around the time you are starting this process?
How many uncommited messages max for the qmgr ?
What is the max msg size for the queue ?
What is the max msg size for the channels you are using ?
 |
|
| Back to top |
|
|
| scott9 |
| Posted: Thu Apr 28, 2005 6:10 pm Post subject: |
|
|
Acolyte
Joined: 11 Jul 2002
Posts: 62
Location: Sacramento,CA
|
All good questions...thanks for taking the time.
> Anyway what are you really trying to do:?
-trying to send a 900MB file through MQ. I believe that the infrastructure is configured to handle this volume. The qmgr stops creating secondary logs after the 50th log is created. I'm trying to understand why additional secondary logs are not created when there is plenty of space available and I have 62 logs configured.
> How many messages are you putting to the queue?
-The application parses a 900MB file into small chunks and places each chunk on a message queue. ~195 messages reach the xmitq before the xaction is rolled-back.
> How big are the messages ?
- The messages are all placed as a single unit of work, presumably under the 4 MB limit. It is a stable application that has not changed in several years. The files are usually ~300MB...this time it is significantly larger.
> How many uncommitted messages on average around the time you are starting this process?
- There are usually zero uncommitted messages at the start. It reaches about 195 when the transaction rolls-back.
> How many uncommited messages max for the qmgr ?
- 10,000
> What is the max msg size for the queue ?
- 4MB
> What is the max msg size for the channels you are using ?
- 4MB |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Apr 28, 2005 7:48 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
OK
You have the right setup but do you have the physical space?
(unix df -k . or df -m .)
You need enough space for the logs in the file system.
You need enough space for the queue in the file system.
You need enough space on the receiving queue in the file system
You need enough space on the receiving log in the file system.
Basically you need 2 * the space (queue + log).
And if the receiving queue manager is on the same box you need 4 times the space. For smaller files you may well get past and the physical space limit may only hit you on these extremes. Remember that once allocated for the queue it takes some time for the qmgr to deallocate again after the need is gone.
And last but not least check out the manuals to see if there is a max size for an open transaction. You might have to split the file before shipping it...
Enjoy  |
|
| Back to top |
|
|
| scott9 |
| Posted: Tue May 03, 2005 1:35 pm Post subject: |
|
|
Acolyte
Joined: 11 Jul 2002
Posts: 62
Location: Sacramento,CA
|
I learned from IBM that MQ will stop creating more logs when the space reaches 80% and the qmgr determines that there is not enough log space to continue. This answers my question about why the qmgr would not allocate more secondary logs, even when there is space for more logs.
Anyway, there is an undocumented tuning parameter that can be added to the qm.ini file to change this value (currently 80%). This comes with a caution that they (IBM) do not recommend changing the default value. I'm posting it here with the same caution.
TuningParameter:
LogThresholdPercentage=90
[Caution: The above parameter overrides the MQ default. Changing this value could adversely affect the qmgr.] |
|
| Back to top |
|
|
| scott9 |
| Posted: Thu May 26, 2005 8:40 am Post subject: RESOLVED: Increased LogFilePages |
|
|
Acolyte
Joined: 11 Jul 2002
Posts: 62
Location: Sacramento,CA
|
...just to close this loop...
I resolved the problem by recreating the qmgr and increasing the LogFilePages. This issue has reminded me how important it is to configure qmgrs with a large value for LogFilePages upfront. It is much easier to ADD more logs later-on, than to DELETE and RECREATE a qmgr when requirements change.
-Cheers! |
|
| Back to top |
|
|
|
|
