| Author |
Message
|
| KIT_INC |
 Posted: Thu Nov 16, 2017 8:37 am Post subject: control maxmsgl of cluster Q |
 |
|
Knight
Joined: 25 Aug 2006
Posts: 589
|
using MQ V8 and MQ V9
I have a cluster of 2 Qmgrs. QM1, QM2, QM3. A cluster Q CLQ1 is defined on QM2 and QM3. QM1 is the MQ Putter. When QM1 put message to CLQ1, the message goes to SYSTEM.CLUSTER.TRANSMIT.QUEUE (SCTQ) of QM1 first and forwarded to QM2 and QM3. Our environment handles a lot of large messages, so all our channels and XMITQs are 100M. CLQ1 has a maxmsgl of 4M because the application using it cannot handle anything bigger. When QM1 accidently PUT message bigger than 4M to the CLQ1, The message goes to DLQ of QM2 or QM3.
Owner of QM2 and QM3 is complaining about this QM1 application causing them issue. They want us, owner of QM1, to handle our own problem. I love to see the application failed with a RC when putting out messages bigger than 4M. But I got my hands tied because the message goes directly to SCTQ which has 100M maxmsgl to support other applications. Unfortunately Alias Q does not have maxmsgl. Is there any other suggestion ? |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Thu Nov 16, 2017 10:17 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
If I understand this...
The consuming app cannot get a message >4M. So, you have set the destination queue MaxMsgLength to 4M.
Along comes an app that MQPUTs a message bigger than 4M, and of course it ends up in the DLQ.
The destination queue MaxMsgLength should be set to the highest value of any message that is supposed to go there.
And, you believe that you can fix this administratively?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| exerk |
| Posted: Thu Nov 16, 2017 10:25 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
I can see no way of preventing the messages going to the DLQ because the application is at fault (as far as I am concerned) - I don't see how one application can 'accidentally' put messages larger than the other application can handle.
For me, this is a good candidate for segmenting the infrastructure, i.e. class-of-service clustering where there is one cluster for 100MB messages and another for 4MB messages, with separate cluster channels and cluster transmission queues for each, but of course that is predicated on the putting application being 'fixed'
With the facility (in the versions you're using) to create separate cluster transmission queues, and especially as they can be auto-created by channels (loose terminology before anyone starts yelling, and I wish that they were not all prefixed SYSTEM.CLUSTER etc.), I now see absolutely no reason for the S.C.T.Q to be used any more.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Nov 16, 2017 10:43 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| exerk wrote: |
| I can see no way of preventing the messages going to the DLQ because the application is at fault (as far as I am concerned) - I don't see how one application can 'accidentally' put messages larger than the other application can handle. |
Seems like the application specs at the requester end have changed, without considering corresponding changes at the responder end.
This is not an MQ cluster issue
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Nov 16, 2017 10:55 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| exerk wrote: |
| I can see no way of preventing the messages going to the DLQ because the application is at fault (as far as I am concerned) - I don't see how one application can 'accidentally' put messages larger than the other application can handle. |
If you need to "handle your own problem" (and my hackles rise at the thought of an MQ administrator being held accountable for the antics of applications using his queue manager), move the errant application to a client connection with a max msg length of 4Mb. This will give you the " application failed with a RC when putting out messages bigger than 4M" that you're looking for.
But the key, root problem is that one application is sending messages the intended target can't process. We assume that these messages have some business significance and are not JPGs of the developer's cat. Therefore whatever these messages are supposed to do to further the generation of revenue is not happening and the application people should be called to task for that.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Nov 16, 2017 10:57 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
If they are JPGs of the developer's cat, move the application from an MQ, message based paradigm to something REST based where the application can exploit the APIs Facebook exposes.
Then fire him.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Nov 16, 2017 11:07 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Moving this to the 'flawed application design' forum... wait... ok... General Discussion forum.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Nov 16, 2017 11:16 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| bruce2359 wrote: |
| I didn't know that Schrodinger had a cat |
Ok, so we won't know if they're JPGs of a cat until we look at them.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| exerk |
| Posted: Thu Nov 16, 2017 11:48 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| Vitor wrote: |
| bruce2359 wrote: |
| I didn't know that Schrodinger had a cat |
Ok, so we won't know if they're JPGs of a cat until we look at them. |
But by observing the cat it may turn into a dog - the action of observing changing that which is being observed...
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Thu Nov 16, 2017 2:46 pm Post subject: Re: control maxmsgl of cluster Q |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| KIT_INC wrote: |
| ...Owner of QM2 and QM3 is complaining about this QM1 application causing them issue. They want us, owner of QM1, to handle our own problem. I love to see the application failed with a RC when putting out messages bigger than 4M. But I got my hands tied because the message goes directly to SCTQ which has 100M maxmsgl to support other applications. Unfortunately Alias Q does not have maxmsgl. Is there any other suggestion ? |
MQ is infrastructure, so it should be configured to handle whatever message routing and queueing is required, with as few limitations as possible. I used to think that maxmsgln should be set to the maximum that the consuming apps can handle, but this is actually a flawed approach. MQ should be responsible for successful delivery of messages, and not an application gate keeper on message length. After a few major production issues due to message length just creeping up over the maxmsgln (typically 4MB), we now have set all our queues and channels to 100MB.
If an app can't consume a message that was sent by another app, it is not MQ's fault, and the issue shouldn't end up in the MQ space. It is a data issue which needs to be resolved by the apps concerned.
If very large message lead to capacity or performance issues, then it can be investigated from an MQ perspective.
_________________
Glenn |
|
| Back to top |
|
|
| tczielke |
| Posted: Thu Nov 16, 2017 3:09 pm Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
Regarding MQ administrative options, one option is to create a second cluster for supporting smaller message lengths and move CLQ1 to this new cluster.
_________________
Working with MQ since 2010. |
|
| Back to top |
|
|
| exerk |
| Posted: Thu Nov 16, 2017 3:39 pm Post subject: Re: control maxmsgl of cluster Q |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| gbaddeley wrote: |
| ...I used to think that maxmsgln should be set to the maximum that the consuming apps can handle, but this is actually a flawed approach... |
Here I would respectfully disagree - the infrastructure can only be sized for what the business will pay for, and that's usually on what they tell you plus an allowance for future growth. If they can't give me the figures they get told they'll get a massive overspec, and they'll have to pay - that normally concentrates the mind!
| gbaddeley wrote: |
| ..MQ should be responsible for successful delivery of messages, and not an application gate keeper on message length... |
Yes and no - it should not be a gatekeeper but over the years I have used maxmsgl to ensure that I know when the business is increasing the stated max without telling me.
| gbaddeley wrote: |
| ...After a few major production issues due to message length just creeping up over the maxmsgln (typically 4MB), we now have set all our queues and channels to 100MB... |
See above - I usually have managed to catch the devious bastards in a lower environment before they hit Prod, and then have been able to ensure Prod had the capacity, or the necessary changes were made.
| gbaddeley wrote: |
| ...If an app can't consume a message that was sent by another app, it is not MQ's fault, and the issue shouldn't end up in the MQ space. It is a data issue which needs to be resolved by the apps concerned... |
Absolutely agree!
| gbaddeley wrote: |
| ...If very large message lead to capacity or performance issues, then it can be investigated from an MQ perspective. |
But so much better to catch it before it happens 
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| exerk |
| Posted: Thu Nov 16, 2017 3:39 pm Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| tczielke wrote: |
| Regarding MQ administrative options, one option is to create a second cluster for supporting smaller message lengths and move CLQ1 to this new cluster. |
But it doesn't stop the app from trying to put bigger messages than the cluster can carry...
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Nov 16, 2017 4:58 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
If the app putting to QM1 is an MQ Client app, set the Max Message Length of the SVRCONN channel to 4MB.
Or,
If the app that puts to QM1 can't be an MQ Client, and there are no other apps on the queue manger that need to send > 4MB, set Max Message Length of the QM to 4MB.
Or,
If QM1 needs to send messages bigger than 4MB to other queues, then revoke the problem app's access to QM1, build QM1B, set the Queue Manager Max Message Length to QM1B to 4 MB, add QM1B to the cluster.
Or,
Create a customer API Exit or Channel Exit to reject these messages.
I'm outta ideas of how to solve for this in the MQ layer.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| tczielke |
| Posted: Thu Nov 16, 2017 7:20 pm Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
| exerk wrote: |
| tczielke wrote: |
| Regarding MQ administrative options, one option is to create a second cluster for supporting smaller message lengths and move CLQ1 to this new cluster. |
But it doesn't stop the app from trying to put bigger messages than the cluster can carry... |
The OP mentioned they were using MQ v8 or v9. With a secondary cluster X and CLQ1 moved to this new cluster X, you could then define a separate SYSTEM.CLUSTER.TRANSMIT.QUEUE on QM1 (multiple cluster transmission queues are allowed at MQ 8.0 and higher) with a MAXMSGL of 4Mb and route cluster work for X through this separate cluster transmission queue. When the app tries to put a message bigger than 4Mb for CLQ1 on QM1, it will now get an error returned to it (2030).
_________________
Working with MQ since 2010. |
|
| Back to top |
|
|
|
|
