ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » General IBM MQ Support » control maxmsgl of cluster Q

Post new topic  Reply to topic Goto page 1, 2  Next
 control maxmsgl of cluster Q « View previous topic :: View next topic » 
Author Message
KIT_INC
PostPosted: Thu Nov 16, 2017 8:37 am    Post subject: control maxmsgl of cluster Q Reply with quote

Knight

Joined: 25 Aug 2006
Posts: 589

using MQ V8 and MQ V9
I have a cluster of 2 Qmgrs. QM1, QM2, QM3. A cluster Q CLQ1 is defined on QM2 and QM3. QM1 is the MQ Putter. When QM1 put message to CLQ1, the message goes to SYSTEM.CLUSTER.TRANSMIT.QUEUE (SCTQ) of QM1 first and forwarded to QM2 and QM3. Our environment handles a lot of large messages, so all our channels and XMITQs are 100M. CLQ1 has a maxmsgl of 4M because the application using it cannot handle anything bigger. When QM1 accidently PUT message bigger than 4M to the CLQ1, The message goes to DLQ of QM2 or QM3.
Owner of QM2 and QM3 is complaining about this QM1 application causing them issue. They want us, owner of QM1, to handle our own problem. I love to see the application failed with a RC when putting out messages bigger than 4M. But I got my hands tied because the message goes directly to SCTQ which has 100M maxmsgl to support other applications. Unfortunately Alias Q does not have maxmsgl. Is there any other suggestion ?
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Nov 16, 2017 10:17 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

If I understand this...

The consuming app cannot get a message >4M. So, you have set the destination queue MaxMsgLength to 4M.

Along comes an app that MQPUTs a message bigger than 4M, and of course it ends up in the DLQ.

The destination queue MaxMsgLength should be set to the highest value of any message that is supposed to go there.

And, you believe that you can fix this administratively?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
exerk
PostPosted: Thu Nov 16, 2017 10:25 am    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

I can see no way of preventing the messages going to the DLQ because the application is at fault (as far as I am concerned) - I don't see how one application can 'accidentally' put messages larger than the other application can handle.

For me, this is a good candidate for segmenting the infrastructure, i.e. class-of-service clustering where there is one cluster for 100MB messages and another for 4MB messages, with separate cluster channels and cluster transmission queues for each, but of course that is predicated on the putting application being 'fixed'

With the facility (in the versions you're using) to create separate cluster transmission queues, and especially as they can be auto-created by channels (loose terminology before anyone starts yelling, and I wish that they were not all prefixed SYSTEM.CLUSTER etc.), I now see absolutely no reason for the S.C.T.Q to be used any more.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Nov 16, 2017 10:43 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

exerk wrote:
I can see no way of preventing the messages going to the DLQ because the application is at fault (as far as I am concerned) - I don't see how one application can 'accidentally' put messages larger than the other application can handle.



Seems like the application specs at the requester end have changed, without considering corresponding changes at the responder end.

This is not an MQ cluster issue
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
Vitor
PostPosted: Thu Nov 16, 2017 10:55 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

exerk wrote:
I can see no way of preventing the messages going to the DLQ because the application is at fault (as far as I am concerned) - I don't see how one application can 'accidentally' put messages larger than the other application can handle.




If you need to "handle your own problem" (and my hackles rise at the thought of an MQ administrator being held accountable for the antics of applications using his queue manager), move the errant application to a client connection with a max msg length of 4Mb. This will give you the " application failed with a RC when putting out messages bigger than 4M" that you're looking for.

But the key, root problem is that one application is sending messages the intended target can't process. We assume that these messages have some business significance and are not JPGs of the developer's cat. Therefore whatever these messages are supposed to do to further the generation of revenue is not happening and the application people should be called to task for that.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Vitor
PostPosted: Thu Nov 16, 2017 10:57 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

If they are JPGs of the developer's cat, move the application from an MQ, message based paradigm to something REST based where the application can exploit the APIs Facebook exposes.

Then fire him.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Nov 16, 2017 11:07 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

Moving this to the 'flawed application design' forum... wait... ok... General Discussion forum.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
Vitor
PostPosted: Thu Nov 16, 2017 11:16 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

bruce2359 wrote:
I didn't know that Schrodinger had a cat


Ok, so we won't know if they're JPGs of a cat until we look at them.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
exerk
PostPosted: Thu Nov 16, 2017 11:48 am    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

Vitor wrote:
bruce2359 wrote:
I didn't know that Schrodinger had a cat


Ok, so we won't know if they're JPGs of a cat until we look at them.

But by observing the cat it may turn into a dog - the action of observing changing that which is being observed...
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Thu Nov 16, 2017 2:46 pm    Post subject: Re: control maxmsgl of cluster Q Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

KIT_INC wrote:
...Owner of QM2 and QM3 is complaining about this QM1 application causing them issue. They want us, owner of QM1, to handle our own problem. I love to see the application failed with a RC when putting out messages bigger than 4M. But I got my hands tied because the message goes directly to SCTQ which has 100M maxmsgl to support other applications. Unfortunately Alias Q does not have maxmsgl. Is there any other suggestion ?

MQ is infrastructure, so it should be configured to handle whatever message routing and queueing is required, with as few limitations as possible. I used to think that maxmsgln should be set to the maximum that the consuming apps can handle, but this is actually a flawed approach. MQ should be responsible for successful delivery of messages, and not an application gate keeper on message length. After a few major production issues due to message length just creeping up over the maxmsgln (typically 4MB), we now have set all our queues and channels to 100MB.

If an app can't consume a message that was sent by another app, it is not MQ's fault, and the issue shouldn't end up in the MQ space. It is a data issue which needs to be resolved by the apps concerned.

If very large message lead to capacity or performance issues, then it can be investigated from an MQ perspective.
_________________
Glenn
Back to top
View user's profile Send private message
tczielke
PostPosted: Thu Nov 16, 2017 3:09 pm    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

Regarding MQ administrative options, one option is to create a second cluster for supporting smaller message lengths and move CLQ1 to this new cluster.
_________________
Working with MQ since 2010.
Back to top
View user's profile Send private message
exerk
PostPosted: Thu Nov 16, 2017 3:39 pm    Post subject: Re: control maxmsgl of cluster Q Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

gbaddeley wrote:
...I used to think that maxmsgln should be set to the maximum that the consuming apps can handle, but this is actually a flawed approach...

Here I would respectfully disagree - the infrastructure can only be sized for what the business will pay for, and that's usually on what they tell you plus an allowance for future growth. If they can't give me the figures they get told they'll get a massive overspec, and they'll have to pay - that normally concentrates the mind!

gbaddeley wrote:
..MQ should be responsible for successful delivery of messages, and not an application gate keeper on message length...

Yes and no - it should not be a gatekeeper but over the years I have used maxmsgl to ensure that I know when the business is increasing the stated max without telling me.

gbaddeley wrote:
...After a few major production issues due to message length just creeping up over the maxmsgln (typically 4MB), we now have set all our queues and channels to 100MB...

See above - I usually have managed to catch the devious bastards in a lower environment before they hit Prod, and then have been able to ensure Prod had the capacity, or the necessary changes were made.

gbaddeley wrote:
...If an app can't consume a message that was sent by another app, it is not MQ's fault, and the issue shouldn't end up in the MQ space. It is a data issue which needs to be resolved by the apps concerned...

Absolutely agree!

gbaddeley wrote:
...If very large message lead to capacity or performance issues, then it can be investigated from an MQ perspective.

But so much better to catch it before it happens
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
exerk
PostPosted: Thu Nov 16, 2017 3:39 pm    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

tczielke wrote:
Regarding MQ administrative options, one option is to create a second cluster for supporting smaller message lengths and move CLQ1 to this new cluster.

But it doesn't stop the app from trying to put bigger messages than the cluster can carry...
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Thu Nov 16, 2017 4:58 pm    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

If the app putting to QM1 is an MQ Client app, set the Max Message Length of the SVRCONN channel to 4MB.

Or,
If the app that puts to QM1 can't be an MQ Client, and there are no other apps on the queue manger that need to send > 4MB, set Max Message Length of the QM to 4MB.

Or,
If QM1 needs to send messages bigger than 4MB to other queues, then revoke the problem app's access to QM1, build QM1B, set the Queue Manager Max Message Length to QM1B to 4 MB, add QM1B to the cluster.

Or,
Create a customer API Exit or Channel Exit to reject these messages.


I'm outta ideas of how to solve for this in the MQ layer.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
tczielke
PostPosted: Thu Nov 16, 2017 7:20 pm    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

exerk wrote:
tczielke wrote:
Regarding MQ administrative options, one option is to create a second cluster for supporting smaller message lengths and move CLQ1 to this new cluster.

But it doesn't stop the app from trying to put bigger messages than the cluster can carry...


The OP mentioned they were using MQ v8 or v9. With a secondary cluster X and CLQ1 moved to this new cluster X, you could then define a separate SYSTEM.CLUSTER.TRANSMIT.QUEUE on QM1 (multiple cluster transmission queues are allowed at MQ 8.0 and higher) with a MAXMSGL of 4Mb and route cluster work for X through this separate cluster transmission queue. When the app tries to put a message bigger than 4Mb for CLQ1 on QM1, it will now get an error returned to it (2030).
_________________
Working with MQ since 2010.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » General IBM MQ Support » control maxmsgl of cluster Q
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.