| Author |
Message
|
| vardhanr8 |
 Posted: Thu Oct 30, 2014 10:59 am Post subject: Impact of increasing the MAXMSGL parm of a receiver channel |
 |
|
Newbie
Joined: 30 Oct 2014
Posts: 1
|
What is the impact of increasing the MAXMSGL parm of a receiver channel?" Does it automatically increase the amount of memory allocated for the channel, regardless of the size of the messages that flow across the channel? For a cluster-receiver channel, which typically supports multiple channel instances, does it increase the memory allocation for each channel instances? (example: if the channel is supporting 10 connections, and we increase MAXMSGL from 4MB to 100MB, does it increase memory usage from 40MB to 1GB?
We have a requirement where app team will send messages of size from 70 MB-100MB.
We are using MQ v7.5.0.3 on AIX.
Thanks!! |
|
| Back to top |
|
 |
| JosephGramig |
| Posted: Thu Oct 30, 2014 11:56 am Post subject: |
|
|
Grand Master
Joined: 09 Feb 2006
Posts: 1244
Location: Gold Coast of Florida, USA
|
First if this is all distributed MQ, use message segmentation which is an option that will auto break the msg in to chunks and reassemble on MQGET (if you specify the option on the PUT and GET).
Next if you do this on a cluster receiver, remember that is the template that the auto cluster senders use for their instances. This means every Qmgr in the cluster needs to be altered to allow this increase. Not just the Qmgr with the cluster receiver. Also you need to increase the XMITQs that go with the cluster senders (and it might not be SYSTEM.CLUSTER.TRANSMIT.QUEUE).
Your buffer is MAXMSGL*BATCHSZ for each channel instance. Make sure this will fit in one log file. A log file size is LogFilePages * 4k (and can only be set when creating the Qmgr (use 16384 if you don't know what it should be)).
If this is UAT or PROD, make LogBufferPages=4096 which is max. |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Thu Oct 30, 2014 12:04 pm Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
To answer the original question though.....no changing the MAXMSGL value will not immediately cause the channel to use more memory. For example, setting it to 100MB will not cause the channel to allocate 100MB....just in case. Of course, by setting this value on a receiver channel, you are opening yourself up to the possibility that someone will send you a 100MB mesage. If that happens then naturally the channel will have to allocate sufficient storage to receive it.
Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| JosephGramig |
| Posted: Fri Oct 31, 2014 4:29 am Post subject: |
|
|
Grand Master
Joined: 09 Feb 2006
Posts: 1244
Location: Gold Coast of Florida, USA
|
| PaulClarke wrote: |
To answer the original question though.....no changing the MAXMSGL value will not immediately cause the channel to use more memory. For example, setting it to 100MB will not cause the channel to allocate 100MB....just in case. Of course, by setting this value on a receiver channel, you are opening yourself up to the possibility that someone will send you a 100MB mesage. If that happens then naturally the channel will have to allocate sufficient storage to receive it.
Cheers,
Paul. |
Your the man and now I'm intrigued, so what is the behavior as the channel starts and progresses through a stack of msgs?
Say I have 100 msgs on the XMITQ and they go from 10K and progress to 100MB?
I assume the channel starts the buffer at some size and grows it as needed? And if this is so, I would expect this to change every time I apply maintenance/upgrades. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Oct 31, 2014 4:41 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| JosephGramig wrote: |
| PaulClarke wrote: |
To answer the original question though.....no changing the MAXMSGL value will not immediately cause the channel to use more memory. For example, setting it to 100MB will not cause the channel to allocate 100MB....just in case. Of course, by setting this value on a receiver channel, you are opening yourself up to the possibility that someone will send you a 100MB mesage. If that happens then naturally the channel will have to allocate sufficient storage to receive it.
Cheers,
Paul. |
Your the man and now I'm intrigued, so what is the behavior as the channel starts and progresses through a stack of msgs?
Say I have 100 msgs on the XMITQ and they go from 10K and progress to 100MB? |
The more interesting, at least to me, question is what happens if the messages start at 100MB and progress to 10K. |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Fri Oct 31, 2014 4:43 am Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
Yes, that is correct. If a channel gets MQRC_TRUNCATED_MSG_FAILED then it will increase the size of the buffer and try again. There is also an algorithm to ensure that the channel doesn't hold on to too much storage it doesn't need. Sooner or later the 100MB will be freed if no messages of that size pass through the channel.
I'm not quite sure why you would expect the storage allocation to change as you apply maintenance or upgrades. As far as I can recall the algorithms haven't changed for a number of releases.
Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Oct 31, 2014 7:15 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Paul,
If amqrmppa.exe can have up to 64 threads, meaning up to 64 channels, and each channel can move up to 100 MB, does that mean amqrmppa.exe might need 64 * 100 MB of memory in the worst case scenario?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Fri Oct 31, 2014 7:21 am Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
Yes, that could happen. Yet another reason why 100MB messages are bad.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| JosephGramig |
| Posted: Fri Oct 31, 2014 7:28 am Post subject: |
|
|
Grand Master
Joined: 09 Feb 2006
Posts: 1244
Location: Gold Coast of Florida, USA
|
Paul,
About maintenance changing the algorithm, I just mean it could and without notice. Say today it starts with a 128k buffer, applying maintenance might change it to 512K. Not that I'm saying I know the algorithm.
Is there a reason why z/OS does not support msg segmentation? |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Oct 31, 2014 7:29 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| PeterPotkay wrote: |
Paul,
If amqrmppa.exe can have up to 64 threads, meaning up to 64 channels, and each channel can move up to 100 MB, does that mean amqrmppa.exe might need 64 * 100 MB of memory in the worst case scenario? |
RAM is cheap.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Oct 31, 2014 7:32 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| bruce2359 wrote: |
| PeterPotkay wrote: |
Paul,
If amqrmppa.exe can have up to 64 threads, meaning up to 64 channels, and each channel can move up to 100 MB, does that mean amqrmppa.exe might need 64 * 100 MB of memory in the worst case scenario? |
RAM is cheap. |
And finite.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Andyh |
| Posted: Fri Oct 31, 2014 7:39 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 239
|
There's no need for a transaction to be contained within a single recovery log extent.
There's a suggestion earlier in this thread that the recovery log extent size should be at least MAXMSGL*BATCHSZ, what is the basis for this suggestion ?
Thanks
Andy. |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Fri Oct 31, 2014 8:36 am Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
Joseph,
Well, it is true that everytime you apply maintenance or install a new version that the memory characteristics may have changed. In fact, I'll venture to suggest that they definitely have. Everytime any change is made to software there is a change to its RAM usage to a greater or lesser extent. So, I'm not quite sure why you are singling out an MCA's memory usage compared to its MAXMSGL parameter rather than, say, it's overall memory usage, or the number of thread stacks or the amount of RAM dedicated to each queue etc etc. As I said, this particular algorithm hasn't changed much for over a decade.
However, it is precisely because things do change that it is always recommended, whereever possible, to test a new level before putting it into production. Most of the time the changes are for the better and the algorithms more sophisticated but from time to time it does cause problems.
As for z/OS not supporting message Segmentation that is not really a question for me to answer. Personally I always think it a shame whenever there is a difference between the z/OS and Distributed products but it usually just comes down to priorities. When Distributed did Message Segmentation I believe z/OS did Shared Queue. I suspect most would agree that Shared Queue had the greater need.
Cheers
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| tczielke |
| Posted: Fri Oct 31, 2014 8:47 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
| PeterPotkay wrote: |
| bruce2359 wrote: |
| PeterPotkay wrote: |
Paul,
If amqrmppa.exe can have up to 64 threads, meaning up to 64 channels, and each channel can move up to 100 MB, does that mean amqrmppa.exe might need 64 * 100 MB of memory in the worst case scenario? |
RAM is cheap. |
And finite. |
If your queue manager is running as a 64 bit application, that finite virtual storage is quite expansive! Also, virtual storage on modern operating systems that I am aware of (i.e. Linux) don't have to be backed by RAM or physical memory (via demand paging) until you actually start reading the virtual storage or putting data into it. So there is some other smoke and mirrors that the OS is doing with virtual storage. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Oct 31, 2014 9:37 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| JosephGramig wrote: |
| Is there a reason why z/OS does not support msg segmentation? |
Message segmentation addresses issues related to limited storage (RAM) more frequently encountered on mid-range platforms, namely: message too big for the queue, the qmgr, the channel, the application (message buffer).
z/OS images are usually provisioned with dramatically (sometimes exponentially) larger central-storage (RAM-equivalent), and thus, lesser need for message segmentation. Initial values for such things as max msg length and max queue depth on z/OS are way bigger on z/OS MQ, reflecting the abundance of both physical and virtual resources.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
|
|
