ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexIBM MQ Performance MonitoringImpact of increasing the MAXMSGL parm of a receiver channel

Post new topicReply to topic Goto page 1, 2  Next
Impact of increasing the MAXMSGL parm of a receiver channel View previous topic :: View next topic
Author Message
vardhanr8
PostPosted: Thu Oct 30, 2014 10:59 am Post subject: Impact of increasing the MAXMSGL parm of a receiver channel Reply with quote

Newbie

Joined: 30 Oct 2014
Posts: 1

What is the impact of increasing the MAXMSGL parm of a receiver channel?" Does it automatically increase the amount of memory allocated for the channel, regardless of the size of the messages that flow across the channel? For a cluster-receiver channel, which typically supports multiple channel instances, does it increase the memory allocation for each channel instances? (example: if the channel is supporting 10 connections, and we increase MAXMSGL from 4MB to 100MB, does it increase memory usage from 40MB to 1GB?

We have a requirement where app team will send messages of size from 70 MB-100MB.

We are using MQ v7.5.0.3 on AIX.

Thanks!!
Back to top
View user's profile Send private message
JosephGramig
PostPosted: Thu Oct 30, 2014 11:56 am Post subject: Reply with quote

Grand Master

Joined: 09 Feb 2006
Posts: 1201
Location: Derby City, USA

First if this is all distributed MQ, use message segmentation which is an option that will auto break the msg in to chunks and reassemble on MQGET (if you specify the option on the PUT and GET).

Next if you do this on a cluster receiver, remember that is the template that the auto cluster senders use for their instances. This means every Qmgr in the cluster needs to be altered to allow this increase. Not just the Qmgr with the cluster receiver. Also you need to increase the XMITQs that go with the cluster senders (and it might not be SYSTEM.CLUSTER.TRANSMIT.QUEUE).

Your buffer is MAXMSGL*BATCHSZ for each channel instance. Make sure this will fit in one log file. A log file size is LogFilePages * 4k (and can only be set when creating the Qmgr (use 16384 if you don't know what it should be)).

If this is UAT or PROD, make LogBufferPages=4096 which is max.
Back to top
View user's profile Send private message AIM Address
PaulClarke
PostPosted: Thu Oct 30, 2014 12:04 pm Post subject: Reply with quote

Guardian

Joined: 17 Nov 2005
Posts: 903
Location: New Zealand

To answer the original question though.....no changing the MAXMSGL value will not immediately cause the channel to use more memory. For example, setting it to 100MB will not cause the channel to allocate 100MB....just in case. Of course, by setting this value on a receiver channel, you are opening yourself up to the possibility that someone will send you a 100MB mesage. If that happens then naturally the channel will have to allocate sufficient storage to receive it.

Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com
Back to top
View user's profile Send private message Visit poster's website
JosephGramig
PostPosted: Fri Oct 31, 2014 4:29 am Post subject: Reply with quote

Grand Master

Joined: 09 Feb 2006
Posts: 1201
Location: Derby City, USA

PaulClarke wrote:
To answer the original question though.....no changing the MAXMSGL value will not immediately cause the channel to use more memory. For example, setting it to 100MB will not cause the channel to allocate 100MB....just in case. Of course, by setting this value on a receiver channel, you are opening yourself up to the possibility that someone will send you a 100MB mesage. If that happens then naturally the channel will have to allocate sufficient storage to receive it.

Cheers,
Paul.

Your the man and now I'm intrigued, so what is the behavior as the channel starts and progresses through a stack of msgs?

Say I have 100 msgs on the XMITQ and they go from 10K and progress to 100MB?

I assume the channel starts the buffer at some size and grows it as needed? And if this is so, I would expect this to change every time I apply maintenance/upgrades.
Back to top
View user's profile Send private message AIM Address
mqjeff
PostPosted: Fri Oct 31, 2014 4:41 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

JosephGramig wrote:
PaulClarke wrote:
To answer the original question though.....no changing the MAXMSGL value will not immediately cause the channel to use more memory. For example, setting it to 100MB will not cause the channel to allocate 100MB....just in case. Of course, by setting this value on a receiver channel, you are opening yourself up to the possibility that someone will send you a 100MB mesage. If that happens then naturally the channel will have to allocate sufficient storage to receive it.

Cheers,
Paul.

Your the man and now I'm intrigued, so what is the behavior as the channel starts and progresses through a stack of msgs?

Say I have 100 msgs on the XMITQ and they go from 10K and progress to 100MB?


The more interesting, at least to me, question is what happens if the messages start at 100MB and progress to 10K.
Back to top
View user's profile Send private message
PaulClarke
PostPosted: Fri Oct 31, 2014 4:43 am Post subject: Reply with quote

Guardian

Joined: 17 Nov 2005
Posts: 903
Location: New Zealand

Yes, that is correct. If a channel gets MQRC_TRUNCATED_MSG_FAILED then it will increase the size of the buffer and try again. There is also an algorithm to ensure that the channel doesn't hold on to too much storage it doesn't need. Sooner or later the 100MB will be freed if no messages of that size pass through the channel.

I'm not quite sure why you would expect the storage allocation to change as you apply maintenance or upgrades. As far as I can recall the algorithms haven't changed for a number of releases.

Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com
Back to top
View user's profile Send private message Visit poster's website
PeterPotkay
PostPosted: Fri Oct 31, 2014 7:15 am Post subject: Reply with quote

Jedi Council

Joined: 15 May 2001
Posts: 7476

Paul,
If amqrmppa.exe can have up to 64 threads, meaning up to 64 channels, and each channel can move up to 100 MB, does that mean amqrmppa.exe might need 64 * 100 MB of memory in the worst case scenario?
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
PaulClarke
PostPosted: Fri Oct 31, 2014 7:21 am Post subject: Reply with quote

Guardian

Joined: 17 Nov 2005
Posts: 903
Location: New Zealand

Yes, that could happen. Yet another reason why 100MB messages are bad.
_________________
Paul Clarke
MQGem Software
www.mqgem.com
Back to top
View user's profile Send private message Visit poster's website
JosephGramig
PostPosted: Fri Oct 31, 2014 7:28 am Post subject: Reply with quote

Grand Master

Joined: 09 Feb 2006
Posts: 1201
Location: Derby City, USA

Paul,

About maintenance changing the algorithm, I just mean it could and without notice. Say today it starts with a 128k buffer, applying maintenance might change it to 512K. Not that I'm saying I know the algorithm.

Is there a reason why z/OS does not support msg segmentation?
Back to top
View user's profile Send private message AIM Address
bruce2359
PostPosted: Fri Oct 31, 2014 7:29 am Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8215
Location: US: west coast, almost. Otherwise, enroute.

PeterPotkay wrote:
Paul,
If amqrmppa.exe can have up to 64 threads, meaning up to 64 channels, and each channel can move up to 100 MB, does that mean amqrmppa.exe might need 64 * 100 MB of memory in the worst case scenario?

RAM is cheap.
_________________
There are two types of people in this world:
1) Those that can extrapolate from incomplete data
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Fri Oct 31, 2014 7:32 am Post subject: Reply with quote

Jedi Council

Joined: 15 May 2001
Posts: 7476

bruce2359 wrote:
PeterPotkay wrote:
Paul,
If amqrmppa.exe can have up to 64 threads, meaning up to 64 channels, and each channel can move up to 100 MB, does that mean amqrmppa.exe might need 64 * 100 MB of memory in the worst case scenario?

RAM is cheap.

And finite.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
Andyh
PostPosted: Fri Oct 31, 2014 7:39 am Post subject: Reply with quote

Disciple

Joined: 29 Jul 2010
Posts: 158

There's no need for a transaction to be contained within a single recovery log extent.
There's a suggestion earlier in this thread that the recovery log extent size should be at least MAXMSGL*BATCHSZ, what is the basis for this suggestion ?

Thanks
Andy.
Back to top
View user's profile Send private message
PaulClarke
PostPosted: Fri Oct 31, 2014 8:36 am Post subject: Reply with quote

Guardian

Joined: 17 Nov 2005
Posts: 903
Location: New Zealand

Joseph,

Well, it is true that everytime you apply maintenance or install a new version that the memory characteristics may have changed. In fact, I'll venture to suggest that they definitely have. Everytime any change is made to software there is a change to its RAM usage to a greater or lesser extent. So, I'm not quite sure why you are singling out an MCA's memory usage compared to its MAXMSGL parameter rather than, say, it's overall memory usage, or the number of thread stacks or the amount of RAM dedicated to each queue etc etc. As I said, this particular algorithm hasn't changed much for over a decade.

However, it is precisely because things do change that it is always recommended, whereever possible, to test a new level before putting it into production. Most of the time the changes are for the better and the algorithms more sophisticated but from time to time it does cause problems.

As for z/OS not supporting message Segmentation that is not really a question for me to answer. Personally I always think it a shame whenever there is a difference between the z/OS and Distributed products but it usually just comes down to priorities. When Distributed did Message Segmentation I believe z/OS did Shared Queue. I suspect most would agree that Shared Queue had the greater need.

Cheers
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com
Back to top
View user's profile Send private message Visit poster's website
tczielke
PostPosted: Fri Oct 31, 2014 8:47 am Post subject: Reply with quote

Shaman

Joined: 08 Jul 2010
Posts: 749
Location: Illinois, USA

PeterPotkay wrote:
bruce2359 wrote:
PeterPotkay wrote:
Paul,
If amqrmppa.exe can have up to 64 threads, meaning up to 64 channels, and each channel can move up to 100 MB, does that mean amqrmppa.exe might need 64 * 100 MB of memory in the worst case scenario?

RAM is cheap.

And finite.


If your queue manager is running as a 64 bit application, that finite virtual storage is quite expansive! Also, virtual storage on modern operating systems that I am aware of (i.e. Linux) don't have to be backed by RAM or physical memory (via demand paging) until you actually start reading the virtual storage or putting data into it. So there is some other smoke and mirrors that the OS is doing with virtual storage.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Fri Oct 31, 2014 9:37 am Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 8215
Location: US: west coast, almost. Otherwise, enroute.

JosephGramig wrote:
Is there a reason why z/OS does not support msg segmentation?

Message segmentation addresses issues related to limited storage (RAM) more frequently encountered on mid-range platforms, namely: message too big for the queue, the qmgr, the channel, the application (message buffer).

z/OS images are usually provisioned with dramatically (sometimes exponentially) larger central-storage (RAM-equivalent), and thus, lesser need for message segmentation. Initial values for such things as max msg length and max queue depth on z/OS are way bigger on z/OS MQ, reflecting the abundance of both physical and virtual resources.
_________________
There are two types of people in this world:
1) Those that can extrapolate from incomplete data
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum IndexIBM MQ Performance MonitoringImpact of increasing the MAXMSGL parm of a receiver channel
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.