|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
 |
|
High number of msgs and LogBufferPages |
« View previous topic :: View next topic » |
Author |
Message
|
mqrules |
Posted: Tue Sep 06, 2022 1:55 pm Post subject: High number of msgs and LogBufferPages |
|
|
Centurion
Joined: 01 Jun 2005 Posts: 100 Location: US
|
Hi All,
Platform: MQ 9.2.0.5 on Linux
QM1 --> QM2 on server1--> QM3 on server1
Problem: When there is a huge number of msgs being sent from QM1 (belonging to another company), msgs accumulate on the XMITQ of QM2 and the curdepth of the target local queue (with Maxdeph 10K) shows 0-10 msgs only...IPPROCS showing 5.
LogBufferPages= 0|0-4096
The amount of memory allocated to buffer records for writing,
specifying the size of the buffers in units of 4 KB pages.
The minimum number of buffer pages is 18 and the maximum
is 4096. Larger buffers lead to higher throughput, especially for
larger messages.
I think setting LogBufferPages=4096 (or to some big number, like 18 or higher) in the qm.ini of QM3 would enable a faster write. Needless to say, I will have to find out from the LINUX admins the size of the current memory and how much more we would need.
Any comments, please?
TIA
MR |
|
Back to top |
|
 |
gbaddeley |
Posted: Tue Sep 06, 2022 4:09 pm Post subject: |
|
|
 Jedi Knight
Joined: 25 Mar 2003 Posts: 2538 Location: Melbourne, Australia
|
Quote: |
msgs accumulate on the XMITQ |
There are many possible reasons for this. Check the throughput on the channel to QM3. How many msgs/sec and MB/sec is it doing during peak period? _________________ Glenn |
|
Back to top |
|
 |
bruce2359 |
Posted: Wed Sep 07, 2022 6:08 am Post subject: Re: High number of msgs and LogBufferPages |
|
|
 Poobah
Joined: 05 Jan 2008 Posts: 9469 Location: US: west coast, almost. Otherwise, enroute.
|
mqrules wrote: |
Problem: When there is a huge number of msgs being sent from QM1 (belonging to another company), |
Numerically, how many is huge?
mqrules wrote: |
msgs accumulate on the XMITQ of QM2 |
How many messages? Are you experiencing channel errors? Display channel status for the channel servicing the xmitq. Display the results here.
mqrules wrote: |
and the curdepth of the target local queue (with Maxdeph 10K) shows 0-10 msgs only...IPPROCS showing 5. |
So, the 5 message consumers seem to be keeping up with the workload.
What problem are you experiencing? I ask because much of what you tell us sounds like normal behavior, and not a problem.
Are your transactions missing SLA (Service Level Agreement)? Something else?
I usually leave log buffer pages at zero to let the qmgr allocate as needed. _________________ I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
Back to top |
|
 |
mqrules |
Posted: Wed Sep 07, 2022 6:50 am Post subject: |
|
|
Centurion
Joined: 01 Jun 2005 Posts: 100 Location: US
|
We are talking about millions of msgs in a very short period of time (don't remember the exact time) especially at month ends... There is really no SLA issue and no complaints...Yes, the product is working the way it is designed. I know, I know, I almost hear some say "if it ain't broke, don't fix it "...
However, there is also a separate case where the msgs would accumulate QM1 XMITQ (with the channels running) and we would get a midnight call from the external company asking us if we have an issue -- and almost always the response is no... Bandwith of the Network connection is not a problem...
Anyway, my question is general: Why not increase the LogBufferPages from the default 0 to get more throughput as a measure of fine-tunining that would help cases like this?
Thanks,
TIA
MR |
|
Back to top |
|
 |
bruce2359 |
Posted: Wed Sep 07, 2022 7:30 am Post subject: |
|
|
 Poobah
Joined: 05 Jan 2008 Posts: 9469 Location: US: west coast, almost. Otherwise, enroute.
|
mqrules wrote: |
Anyway, my question is general: Why not increase the LogBufferPages from the default 0 to get more throughput as a measure of fine-tunining that would help cases like this?
Thanks,
TIA
MR |
Why are you target-locked on logbufferpages? Why do you believe that adjusting this will improve throughput? What else have you tried?
What metric would you use to determine if this tuning effort was successful or not? The external folks calling? Or, not calling? Something else?
What hardware platform is at the originating end? What o/s? What MQ VRMF?
To help us help you, please ... display the channel status of the sender/receiver channel that services the xmitq, and post the results here? Best done when throughput appears to be a problem for you.
Any channel error logged in AMQERR01 for this qmgr? _________________ I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
Back to top |
|
 |
Andyh |
Posted: Wed Sep 07, 2022 8:44 am Post subject: |
|
|
Master
Joined: 29 Jul 2010 Posts: 239
|
Almost everyone doing any volume of persistent messaging should set LogBufferPages to the maximum supported value. This parameter is specified in units of log pages, which on the distributed MQ product are currently always 4KB. Setting 4096 will therefore consume only 16MB of memory which might have been a lot when MQ was first written (1990's) but is now a trivial amount of memory. The size of the log buffer (16MB at 4096) is the size of the largest single write the queue manager can make to the log. The queue manager marshals message data into the log buffer to allow writes from multiple puts (either concurrent puts or a batch from a single writer) to be flushed to disk in a single synchonous write. Setting LogBufferPages to 4096 eliminates this as a possible restriction and is a trivial change (requires the QMgr to be recycled).
As you report the messages to be building up on a transmit queue you may also want to check that channel batching is enable and to measure the effective batch size being achieved. |
|
Back to top |
|
 |
mqrules |
Posted: Wed Sep 07, 2022 6:13 pm Post subject: |
|
|
Centurion
Joined: 01 Jun 2005 Posts: 100 Location: US
|
Thank you Andyh and others for your valuable comments.
MR |
|
Back to top |
|
 |
|
|
 |
|
Page 1 of 1 |
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|
|
|