| Author |
Message
|
| vmanel |
 Posted: Wed Sep 11, 2019 8:44 am Post subject: Messages in DLQ |
 |
|
Acolyte
Joined: 12 Mar 2019
Posts: 57
|
Hi,
We see that there are few messages in DLQ in production.
The application which has put the message is amqpcsea and the reason is "mqrc_q_deleted" for the destination queue AMQ.5D7133BE2C1ADD5C.
This is not an application queue and this looks like a temporary dynamic queue which got deleted before the reply for the command was sent
We did not run any command on this queue manager, but we are connected to this QM from MQ Explorer, so could it because of some operation performed through MQ Explorer? Are MQ Explorer operations performed via SYSTEM.ADMIN.COMMAND.QUEUE?
The SYSTEM.ADMIN.COMMAND.QUEUE has default persistent as YES. So could this be the reason for the messages to go to DLQ? But the reply messages which has gone to DLQ are non-persistent
Though the persistence property of that queue was changed a year ago, we are seeing this issue first time and that too only few messages are there in DLQ, and this issue was never observed in lower environments |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Wed Sep 11, 2019 1:01 pm Post subject: Re: Messages in DLQ |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| vmanel wrote: |
Hi,
We see that there are few messages in DLQ in production.
The application which has put the message is amqpcsea and the reason is "mqrc_q_deleted" for the destination queue AMQ.5D7133BE2C1ADD5C.
This is not an application queue and this looks like a temporary dynamic queue which got deleted before the reply for the command was sent
We did not run any command on this queue manager, but we are connected to this QM from MQ Explorer, so could it because of some operation performed through MQ Explorer? Are MQ Explorer operations performed via SYSTEM.ADMIN.COMMAND.QUEUE?
The SYSTEM.ADMIN.COMMAND.QUEUE has default persistent as YES. So could this be the reason for the messages to go to DLQ? But the reply messages which has gone to DLQ are non-persistent
Though the persistence property of that queue was changed a year ago, we are seeing this issue first time and that too only few messages are there in DLQ, and this issue was never observed in lower environments |
There was sometime back in early versions of MQ8 a bug by which the last response message from the command server was not picked up from the requester... maybe that can help 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Sep 11, 2019 2:22 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Any FDC's or errors logged in he error logs?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Wed Sep 11, 2019 3:37 pm Post subject: Re: Messages in DLQ |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| vmanel wrote: |
Hi,
We see that there are few messages in DLQ in production.
The application which has put the message is amqpcsea and the reason is "mqrc_q_deleted" for the destination queue AMQ.5D7133BE2C1ADD5C.
... |
This is the classic situation where an app sent a PCF command message to the MQ command processor, with reply queue name set to a dynamic queue. The app terminated &/or the dynamic queue was deleted before the MQ command processor could put all of the command reply messages. Such is life.
We have a triggered runmqdlq with rules that discard these messages, eg.
ACTION(DISCARD) REASON(2085) DESTQ(WMQFTE.*)
ACTION(DISCARD) REASON(2085) DESTQ(AMQ.*)
ACTION(DISCARD) REASON(2085) DESTQ(MQMON.*)
_________________
Glenn |
|
| Back to top |
|
|
| hughson |
| Posted: Thu Sep 12, 2019 2:19 am Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1959
Location: Bay of Plenty, New Zealand
|
|
| Back to top |
|
|
| HubertKleinmanns |
| Posted: Thu Sep 12, 2019 3:31 am Post subject: Re: Messages in DLQ |
|
|
Shaman
Joined: 24 Feb 2004
Posts: 732
Location: Germany
|
| vmanel wrote: |
| The SYSTEM.ADMIN.COMMAND.QUEUE has default persistent as YES. So could this be the reason for the messages to go to DLQ? |
No, this is the response of your PCF command, which shall be put into the queue "AMQ...", not into the command queue. The queue default persistency is set by the corresponding model queue. And non-persistent messages may put to the DLQ as well.
By the way: A temporary dynamic queue created by MQ Explorer should have a name like "AMQ.MQEXPLORER..." and MQMON (MO71) should have a name like "MQMON.<username>...".
Maybe it's another tool which creates the queue?
_________________
Regards
Hubert |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Sep 12, 2019 5:05 am Post subject: Re: Messages in DLQ |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| HubertKleinmanns wrote: |
| No, this is the response of your PCF command, which shall be put into the queue "AMQ...", not into the command queue. The queue default persistency is set by the corresponding model queue. And non-persistent messages may put to the DLQ as well. |
And an attempt to put a persistent message to a temporary dynamic queue will fail with a reason code I'm too lazy to look up.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| HubertKleinmanns |
| Posted: Thu Sep 12, 2019 5:14 am Post subject: Re: Messages in DLQ |
|
|
Shaman
Joined: 24 Feb 2004
Posts: 732
Location: Germany
|
| Vitor wrote: |
| And an attempt to put a persistent message to a temporary dynamic queue will fail with a reason code I'm too lazy to look up. |
2048 0x00000800 MQRC_PERSISTENT_NOT_ALLOWED 
_________________
Regards
Hubert |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Sep 12, 2019 5:23 am Post subject: Re: Messages in DLQ |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| HubertKleinmanns wrote: |
| Vitor wrote: |
| And an attempt to put a persistent message to a temporary dynamic queue will fail with a reason code I'm too lazy to look up. |
2048 0x00000800 MQRC_PERSISTENT_NOT_ALLOWED |
Thank you for enabling me.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| tczielke |
| Posted: Thu Sep 12, 2019 8:15 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
What is being described by the OP sounds an awful lot like the command server bug where it attempts to PUT another reply after it has closed the reply queue. I still see evidence of this sporadically happening in my MQ environment, even with more recent levels of MQ software. I am not 100% sure, but pretty sure I have seen recent evidence of this happening on MQ 9.1.0.1 queue managers. As fjb_saper mentioned, there have been previous bugs like this that have been fixed in the command server.
_________________
Working with MQ since 2010. |
|
| Back to top |
|
|
|
|
