| Author |
Message
|
| MABeatty1978 |
 Posted: Fri Oct 28, 2016 10:15 am Post subject: Messages stuck n cluster xmit q SYSTEM.CHANNEL.INITQ damage |
 |
|
Acolyte
Joined: 17 Jul 2014
Posts: 54
|
I have a bunch of messages stuck in the cluster tranmission queue. In the logs I see:
-------------------------------------------------------------------------------
10/28/2016 02:07:35 PM - Process(10249.1) User(mqm) Program(runmqchi)
Host(pos4359)
AMQ9509: Program cannot open queue manager object.
EXPLANATION:
The attempt to open either the queue or queue manager object
'SYSTEM.CHANNEL.INITQ' on queue manager 'PROD_POS4359_QM' failed with reason
code 2101.
ACTION:
Ensure that the queue is available and retry the operation.
Any advice how to proceed? |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Fri Oct 28, 2016 11:19 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Does your sender channel definition name SYSTEM.CHANNEL.INITQ as the transmission queue? It shouldn't. The SYSTEM.CHANNEL.INITQ is used as a trigger event queue, and not a transmission queue.
Post your sender channel definition here.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| hughson |
| Posted: Sun Oct 30, 2016 1:58 pm Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1959
Location: Bay of Plenty, New Zealand
|
The error message is written by the Channel Initiator (runmqchi) so I don't think it is being used as a transmission queue. The Channel Initiator is the process that is supposed to be reading the SYSTEM.CHANNEL.INITQ.
Given that the queue is damaged (MQRC 2101) you need to resolve that.
If you have some cluster channels not running (shown by the fact that you have messages stuck in the cluster transmission queue) you can manually start the channels in the mean time until you get your Channel Initiator fixed.
If you don't know what channels need to be started to move the stuck messages, try the command:-
| Code: |
| DISPLAY CHSTATUS(*) WHERE(XQMSGSA > 1) |
Cheers
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Oct 30, 2016 7:16 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| hughson wrote: |
The error message is written by the Channel Initiator (runmqchi) so I don't think it is being used as a transmission queue. The Channel Initiator is the process that is supposed to be reading the SYSTEM.CHANNEL.INITQ.
Given that the queue is damaged (MQRC 2101) you need to resolve that.
If you have some cluster channels not running (shown by the fact that you have messages stuck in the cluster transmission queue) you can manually start the channels in the mean time until you get your Channel Initiator fixed.
If you don't know what channels need to be started to move the stuck messages, try the command:-
| Code: |
| DISPLAY CHSTATUS(*) WHERE(XQMSGSA > 1) |
Cheers
Morag |
Given the fact that the object is damaged it is well possible that the channel won't start.
If you run with linear logging you can recover the object.
Otherwise you may have to delete and recreate the object (and loose the messages awaiting transmission)...
<edit> and loose the messages on the queue (not awaiting transmission)... Thanks Morag
Was focused on SCTQ as I had a case of a bad queue there (but no clustering involved)...</edit>
Have fun 
_________________
MQ & Broker admin
Last edited by fjb_saper on Tue Nov 01, 2016 5:10 am; edited 3 times in total |
|
| Back to top |
|
|
| hughson |
| Posted: Sun Oct 30, 2016 7:46 pm Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1959
Location: Bay of Plenty, New Zealand
|
| fjb_saper wrote: |
| Otherwise you may have to delete and recreate the object (and loose the messages awaiting transmission)... |
There are no messages awaiting transmission on the SYSTEM.CHANNEL.INITQ. The worst thing you can lose is a few trigger messages, and you can manually start the channels that didn't get started.
There are no user messages on the SYSTEM.CHANNEL.INITQ.
Stopping and restarting the queue manager might be a good plan once the object is fixed, to regenerate any missing trigger messages.
Cheers
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| Andyh |
| Posted: Mon Oct 31, 2016 1:48 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 239
|
If a queue is damaged then there should be FDC's detailing what damage has occurred. Unless you've been deploying some very dubious practices (such as manually deleting MQ owned files) then these FDC's should be reported to MQ support and more fully investigated (otherwise who's to say when the issue might next occur, and with what effect).
A damaged object indicates some inconsistency in the objects state, this might be an issue with the state in memory or on disk. 'Simply' restarting the queue manager might resolve the damage ( for example if it was the memory image that was corrupt), or the object might need to be recovered (or deleted and redefined). As there's no 'persistent' state on an INITQ then deleting and redefining shouldn't be significantly different from recovering form linear logs (if applicable). |
|
| Back to top |
|
|
| MABeatty1978 |
| Posted: Mon Oct 31, 2016 5:49 am Post subject: |
|
|
Acolyte
Joined: 17 Jul 2014
Posts: 54
|
| fjb_saper wrote: |
If you run with linear logging you can recover the object.
Otherwise you may have to delete and recreate the object (and loose the messages awaiting transmission)...
Have fun |
Thank you for this. I was unable to determine root cause, but by running rcrmqobj, I was able to recreate the queue upon which all of my cluster channels fired up and the stuck messages were sent on the merry way. |
|
| Back to top |
|
|
|
|
