ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » General IBM MQ Support » control maxmsgl of cluster Q

Post new topic  Reply to topic Goto page Previous  1, 2
 control maxmsgl of cluster Q « View previous topic :: View next topic » 
Author Message
exerk
PostPosted: Fri Nov 17, 2017 1:29 am    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

tczielke wrote:
exerk wrote:
tczielke wrote:
Regarding MQ administrative options, one option is to create a second cluster for supporting smaller message lengths and move CLQ1 to this new cluster.

But it doesn't stop the app from trying to put bigger messages than the cluster can carry...


The OP mentioned they were using MQ v8 or v9. With a secondary cluster X and CLQ1 moved to this new cluster X, you could then define a separate SYSTEM.CLUSTER.TRANSMIT.QUEUE on QM1 (multiple cluster transmission queues are allowed at MQ 8.0 and higher) with a MAXMSGL of 4Mb and route cluster work for X through this separate cluster transmission queue. When the app tries to put a message bigger than 4Mb for CLQ1 on QM1, it will now get an error returned to it (2030).

I alluded to that when I mentioned class-of-service clusters and the OP stated...

Quote:
...Owner of QM2 and QM3 is complaining about this QM1 application causing them issue. They want us, owner of QM1, to handle our own problem. I love to see the application failed with a RC when putting out messages bigger than 4M.

That however assumes the application will be able to recover from such an issue and any business is going to be extremely upset if their application fails and needs restart every time it happens.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
tczielke
PostPosted: Fri Nov 17, 2017 5:26 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

exerk wrote:
tczielke wrote:
exerk wrote:
tczielke wrote:
Regarding MQ administrative options, one option is to create a second cluster for supporting smaller message lengths and move CLQ1 to this new cluster.

But it doesn't stop the app from trying to put bigger messages than the cluster can carry...


The OP mentioned they were using MQ v8 or v9. With a secondary cluster X and CLQ1 moved to this new cluster X, you could then define a separate SYSTEM.CLUSTER.TRANSMIT.QUEUE on QM1 (multiple cluster transmission queues are allowed at MQ 8.0 and higher) with a MAXMSGL of 4Mb and route cluster work for X through this separate cluster transmission queue. When the app tries to put a message bigger than 4Mb for CLQ1 on QM1, it will now get an error returned to it (2030).

I alluded to that when I mentioned class-of-service clusters and the OP stated...

Quote:
...Owner of QM2 and QM3 is complaining about this QM1 application causing them issue. They want us, owner of QM1, to handle our own problem. I love to see the application failed with a RC when putting out messages bigger than 4M.

That however assumes the application will be able to recover from such an issue and any business is going to be extremely upset if their application fails and needs restart every time it happens.


I went back and read this thread, and you have an earlier post that states this:

Quote:
I can see no way of preventing the messages going to the DLQ because the application is at fault (as far as I am concerned) - I don't see how one application can 'accidentally' put messages larger than the other application can handle.

For me, this is a good candidate for segmenting the infrastructure, i.e. class-of-service clustering where there is one cluster for 100MB messages and another for 4MB messages, with separate cluster channels and cluster transmission queues for each, but of course that is predicated on the putting application being 'fixed'

With the facility (in the versions you're using) to create separate cluster transmission queues, and especially as they can be auto-created by channels (loose terminology before anyone starts yelling, and I wish that they were not all prefixed SYSTEM.CLUSTER etc.), I now see absolutely no reason for the S.C.T.Q to be used any more.


That is not how I remember it reading when I read it last evening. I remember it reading more like how two other people quoted it:

Quote:
I can see no way of preventing the messages going to the DLQ because the application is at fault (as far as I am concerned) - I don't see how one application can 'accidentally' put messages larger than the other application can handle


I am not sure what happened (e.g. it was edited after my original post, I missed the comments when I read it last evening, etc.), but that is why I made a comment that is just a more terse reply of what you are stating in your earlier post.
_________________
Working with MQ since 2010.
Back to top
View user's profile Send private message
exerk
PostPosted: Fri Nov 17, 2017 3:41 pm    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

tczielke wrote:
I am not sure what happened (e.g. it was edited after my original post, I missed the comments when I read it last evening, etc.), but that is why I made a comment that is just a more terse reply of what you are stating in your earlier post.

It's been a long week for all of us...
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Sun Nov 19, 2017 5:12 pm    Post subject: Re: control maxmsgl of cluster Q Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

exerk wrote:
gbaddeley wrote:
...I used to think that maxmsgln should be set to the maximum that the consuming apps can handle, but this is actually a flawed approach...

Here I would respectfully disagree - the infrastructure can only be sized for what the business will pay for, and that's usually on what they tell you plus an allowance for future growth. If they can't give me the figures they get told they'll get a massive overspec, and they'll have to pay - that normally concentrates the mind!

Mutiplying # app local queues by maxdepth by maxmsgln usually gives a figure in the peta bytes, which is an impossible size of disk. We use disk space monitoring to watch for growth patterns.
exerk wrote:
gbaddeley wrote:
..MQ should be responsible for successful delivery of messages, and not an application gate keeper on message length...

Yes and no - it should not be a gatekeeper but over the years I have used maxmsgl to ensure that I know when the business is increasing the stated max without telling me.

You know because there is a production messaging failure on a critical purchase order message? That's not really a good time to find out.
exerk wrote:
gbaddeley wrote:
...After a few major production issues due to message length just creeping up over the maxmsgln (typically 4MB), we now have set all our queues and channels to 100MB...

See above - I usually have managed to catch the devious bastards in a lower environment before they hit Prod, and then have been able to ensure Prod had the capacity, or the necessary changes were made.

Quite often lower environments do not mimic prod message sizes, particularly edge cases.

A recent example is where a prod source system was down for several hours so it couldn't send orders through MQ even 10 mins. When it came up, it attempted to send all backlogged orders through MQ in one message. Instead of 10K messages we had a 6MB message, which blew the 4MB max length. After scrambling through dead letter queues, we manage to alter the queue maxmsgl and redeliver the message just before the order cutoff time. The business was not happy.
_________________
Glenn
Back to top
View user's profile Send private message
exerk
PostPosted: Mon Nov 20, 2017 2:20 am    Post subject: Re: control maxmsgl of cluster Q Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

gbaddeley wrote:
Mutiplying # app local queues by maxdepth by maxmsgln usually gives a figure in the peta bytes, which is an impossible size of disk. We use disk space monitoring to watch for growth patterns.

But it starts the conversation about what is possible, what monitoring is needed, who is going to pay for it, and gets them in the groove of thinking ahead in regard to growth.

gbaddeley wrote:
You know because there is a production messaging failure on a critical purchase order message? That's not really a good time to find out.

A more disciplined approach generally means that scenario should not occur - the business should not breach the set maximums (but we all know it happens).

gbaddeley wrote:
Quite often lower environments do not mimic prod message sizes, particularly edge cases.

Pre-Production should but I acknowledge there are always exceptions.

gbaddeley wrote:
A recent example is where a prod source system was down for several hours so it couldn't send orders through MQ even 10 mins. When it came up, it attempted to send all backlogged orders through MQ in one message. Instead of 10K messages we had a 6MB message, which blew the 4MB max length. After scrambling through dead letter queues, we manage to alter the queue maxmsgl and redeliver the message just before the order cutoff time. The business was not happy.

Was the business not happy that it was just before cut-off, or not happy that the system was down, or not happy that you had to make changes to expedite the larger message size?

If we lived in a perfect world we'd get all the information from the app owners up-front, it would be accurate, and best of all we'd know the exact profile of the app, including such things as your example above. We'd then be able to plan for such things, size for such things, and hopefully pre-empt and prevent such things, but we don't live in an ideal world.

Wherever I go I try to effect a culture change and move a step closer to ideal, try to get the app people to involve the middle-ware people early, not when all the decisions are made and they "just need infrastructure". While MQ is still considered to be a bit of Cat 5 cable however I fear it is going to be an ongoing, and up-hill, struggle!
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Mon Nov 20, 2017 3:49 pm    Post subject: Re: control maxmsgl of cluster Q Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

exerk wrote:
Was the business not happy that it was just before cut-off, or not happy that the system was down, or not happy that you had to make changes to expedite the larger message size?

Not happy because an arbitrary limit had prevented a valid message (for both the source & consumer apps) from going through.

The choices are:
1/ Let all MQ app messages go through regardless of length. Let the consumer app deal with the consequences.
2/ Quarantine some messages because they are larger than a limit that was set by the MQ admin when the queue was originally created, or inherited from the system default object.
_________________
Glenn
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Mon Nov 20, 2017 4:51 pm    Post subject: Re: control maxmsgl of cluster Q Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

gbaddeley wrote:
exerk wrote:
Was the business not happy that it was just before cut-off, or not happy that the system was down, or not happy that you had to make changes to expedite the larger message size?

Not happy because an arbitrary limit had prevented a valid message (for both the source & consumer apps) from going through.

The choices are:
1/ Let all MQ app messages go through regardless of length. Let the consumer app deal with the consequences.
2/ Quarantine some messages because they are larger than a limit that was set by the MQ admin when the queue was originally created, or inherited from the system default object.



If you do max all the settings you are indeed free to and walk away when they try and send a message that is too big. On the other hand, a smaller limit that they provide initially does allow for the size to be bumped to "be a hero" and does get everyone focused on the topic. To what end? You leave the bigger size, they shrug their shoulders when pressed on why they didn't come up with an accurate Max Size, or proactively tell you to up it as their business and thus message size grew.

Some days I just wish I could make all the queues nine 9s x 100 MB, add 500 GB of storage per Queue Manager and be done with it all. If they won't spend on the people to be able to come up with good requirements, to spend money on proper capacity planning, then throw a few pennies (comparatively speaking) at the infrastructure to oversize it and be done with it.

It would be nice to have a tool that reported on average, min and max message size per queue, per hour/day/week/month plotted it all in graphs for review, with configurable alerts to tell you if your average size is rising over time, or approaching within x% of Max Message Length.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Tue Nov 21, 2017 6:09 am    Post subject: Re: control maxmsgl of cluster Q Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

PeterPotkay wrote:

It would be nice to have a tool that reported on average, min and max message size per queue, per hour/day/week/month plotted it all in graphs for review, with configurable alerts to tell you if your average size is rising over time, or approaching within x% of Max Message Length.


Didn't QPasa do that for you?
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
KIT_INC
PostPosted: Wed Nov 22, 2017 12:16 pm    Post subject: Reply with quote

Knight

Joined: 25 Aug 2006
Posts: 589

Thanks for all the responses. I was tied up with some production issue for a few days. A little bit more on the back ground.
Yes, I totally agreed that App on QM1 is at fault and should be fixed. The history is QM1 is one company and QM2, QM3 belong to another. App on QM1 was running locally. It MQPUT the result of a DB call to a Q. if the result is too big. It got a bad RC and handle that from there. So there was nothing wrong with the APP until the organizational change and the receiving App get moved to QM2 and QM3. One obvious solution is like what was discussed here to use a separate channel with an XMITQ with the right maxmsgl. Then App on QM1 will still get the bad RC as before. But because we are using clustering and remote Q definition is now considered an exception.
Thinking along that line, MQ clustering allow the use of multiple SCTQ. I probably have not finish reading everything. So far I have not find any instruction on how can I tell the App to choose which SCTQ to use. For example, I have 2 clssdr chls from QM1 TO QM2 (100M.QM2 and 4M.MQ2).
100M.QM2 is using SCTQ1 with maxmsgl of 100M while 4M.QM2 is suing SCTQ2 with maxmsgl of 4M. But how can I control which SCTQ to use when APP on QM1 is putting to cluster Q on QM2 and QM3

Just a second thought, it will be easy if there is a maxmsgl attribute for the alias Q definition which MQ will check and fail the App.


Last edited by KIT_INC on Wed Nov 22, 2017 12:19 pm; edited 1 time in total
Back to top
View user's profile Send private message
JosephGramig
PostPosted: Wed Nov 22, 2017 12:17 pm    Post subject: Reply with quote

Grand Master

Joined: 09 Feb 2006
Posts: 1230
Location: Gold Coast of Florida, USA

So this sure sounds like a "General MQ Support" question rather than a "General Discussion" question.

General questions are like: What do I inject the turkey with before I deep fry it? Or, What is the number of the fire department to put out the fire on my deck from the deep fried turkey?
Back to top
View user's profile Send private message AIM Address
exerk
PostPosted: Wed Nov 22, 2017 2:37 pm    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

KIT_INC wrote:
..But how can I control which SCTQ to use when APP on QM1 is putting to cluster Q on QM2 and QM3

Class-of-service clusters, i.e. create a second cluster with just that queue in it.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Wed Nov 22, 2017 4:07 pm    Post subject: Re: control maxmsgl of cluster Q Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

fjb_saper wrote:
PeterPotkay wrote:

It would be nice to have a tool that reported on average, min and max message size per queue, per hour/day/week/month plotted it all in graphs for review, with configurable alerts to tell you if your average size is rising over time, or approaching within x% of Max Message Length.


Didn't QPasa do that for you?


No, because it had no access to the size of the messages. Just the number of them.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Thu Nov 23, 2017 2:20 pm    Post subject: Re: control maxmsgl of cluster Q Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

PeterPotkay wrote:
It would be nice to have a tool that reported on average, min and max message size per queue, per hour/day/week/month plotted it all in graphs for review, with configurable alerts to tell you if your average size is rising over time, or approaching within x% of Max Message Length.

It would be nice if MQ Queue Statistics had metrics related to message length, but alas, it doesn't.
_________________
Glenn
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Fri Nov 24, 2017 4:16 am    Post subject: Re: control maxmsgl of cluster Q Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

PeterPotkay wrote:
fjb_saper wrote:
PeterPotkay wrote:

It would be nice to have a tool that reported on average, min and max message size per queue, per hour/day/week/month plotted it all in graphs for review, with configurable alerts to tell you if your average size is rising over time, or approaching within x% of Max Message Length.


Didn't QPasa do that for you?


No, because it had no access to the size of the messages. Just the number of them.

I thought it had number of bytes... well maybe not for queues but certainly for channels...
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic  Reply to topic Goto page Previous  1, 2 Page 2 of 2

MQSeries.net Forum Index » General IBM MQ Support » control maxmsgl of cluster Q
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.