| Author |
Message
|
| mca |
 Posted: Wed Jan 20, 2016 1:30 pm Post subject: Queue file size limit |
 |
|
Disciple
Joined: 09 Mar 2005
Posts: 196
|
We just converted our WMQ/WMB environment from WMQv7 to WMQv8 & WMBv8 to IIBv9 on Linux. We are in Production processing orders few on old system and rest on new IIB.
Last couple of days, we had production issue on new environment (IIB), with MQ not responsive and it is complaining about "hrcE_LOG_FULL (or) AMQ6709: The log for the queue manager is full" We have abundant dis space. The queues are defined non-persistent on both servers.
We compared our file structure of MQv8 with MQv7 and discovered that the queue files listed under /var/mqm/qmgrs/QMName/queues/ are of fixed size of 4096 (directory) on MQv7 vs. they are random sized and large size files on MQv8.
MQv7: drwxrwx--- 2 mqm mqm 4096 Jan 18 11:05 QueueName
MQv8: -rw-rw---- 1 mqm mqm 139984896 Jan 20 16:25 QueueName
Can someone help us identify where we missed the setting in new version? |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Wed Jan 20, 2016 6:05 pm Post subject: Re: Queue file size limit |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| mca wrote: |
We compared our file structure of MQv8 with MQv7 and discovered that the queue files listed under /var/mqm/qmgrs/QMName/queues/ are of fixed size of 4096 (directory) on MQv7 vs. they are random sized and large size files on MQv8.
MQv7: drwxrwx--- 2 mqm mqm 4096 Jan 18 11:05 QueueName
MQv8: -rw-rw---- 1 mqm mqm 139984896 Jan 20 16:25 QueueName
Can someone help us identify where we missed the setting in new version? |
Obviously you're just being confused here.
MQV7 shows a directorty and MQV8 shows a file.
It looks like when you're working with MQ V7.0.x (BTW out of support) you need to dig one level deeper to get to the queue file....
You need to compare like to like.
By the way does your OS have support for large files? (> 2GB)
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Jan 20, 2016 6:28 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Circular Logs or Linear Logs?
| Quote: |
| The queues are defined non-persistent on both servers. |
Irrelevant. Apps can place persistent messages in queues were the default persistent attribute is Non Persistent. Additionally, non persistent messages do not mean the MQ Logs are not involved - if the apps are doing MQPUTs or MQGETs under syncpoint, the logs are involved whether the message is persistent or not. And if the apps don't commit for a long time, the QM has to keep track of those open Units Of Work (UOW) in the logs, which then grow, and grow, and grow.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Andyh |
| Posted: Thu Jan 21, 2016 2:42 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 239
|
MQ V8 is a little lazier than V7 in flushing data to disk which can result in slightly higher log space requirements. This is then compounded by a bug in early V8 service levels (fixed by APAR) where the queue manager slowed down in reaction to a log space shortage (rather than becoming more proactive in forcing data to disk).
Having lots of disk space in the log file system is only relevant if the recovery logs are defined in a manner that allows them to use this space.
The change from using a directory per queue to a file per queue was in response to limitation on the number of sub-directories per directory supported on some platforms. Earlier releases were capable of being configured in this manner, but V8 defaults to this behaviour.
Recent MQ releases are capable of supporting very large populations of clients/queue where this restriction was becoming a more common issue. |
|
| Back to top |
|
|
| Andyh |
| Posted: Thu Jan 21, 2016 2:47 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 239
|
b.t.w it's not true that non-persistent messages require MQ log records to be written, even if those messages are put/got within transactions.
An MQ transaction involving only non-persistent messages will not write any MQ log records. The transaction 'log' in this case is maintained entirely in memory, if the queue manager should fail the non-persistent messages will be deleted and so no record of these messages is required in the log. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Jan 21, 2016 2:55 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| Andyh wrote: |
b.t.w it's not true that non-persistent messages require MQ log records to be written, even if those messages are put/got within transactions.
An MQ transaction involving only non-persistent messages will not write any MQ log records. The transaction 'log' in this case is maintained entirely in memory, if the queue manager should fail the non-persistent messages will be deleted and so no record of these messages is required in the log. |
So if an app does an MQPUT under syncpoint of a non persistent message, and then just sits there, never ending, never committing or backing out, and the QM continues to run processing huge volumes for other unrelated transactions for other apps, that first app's MQPUT is not holding anything in the transaction log that would cause the QM to eventually force a back out because the log space was filled?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Jan 21, 2016 2:58 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| Andyh wrote: |
Having lots of disk space in the log file system is only relevant if the recovery logs are defined in a manner that allows them to use this space.
|
Do you mean Linear Logs instead of Circular?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Andyh |
| Posted: Thu Jan 21, 2016 10:47 pm Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 239
|
1. Yes, a transaction involving only non-persistent messages doesn't write anything to the recovery log and thus has no implications for asynchronous rollback due to log space usage.
2. Both circular and linear will restrict the size of the active log to the number of primaries + the number of secondaries (actually the QMgr will start to take action to reduce the log space usage when 80% of the space defined by the number of primaries + the number of secondaries is reached). Having lots of further disk space in the file system in which the recovery log is placed doesn't imply a bigger active log. |
|
| Back to top |
|
|
|
|
