| Author |
Message
|
| hp1411 |
 Posted: Tue Aug 23, 2016 8:19 am Post subject: Queue manager quiescing |
 |
|
Newbie
Joined: 17 Oct 2015
Posts: 9
|
Hello,
I am hoping someone can help me out with an issue that I have
I am running Queue managers in High Availability mode
when we try to switch our queue manager to the standby node
endmqm -s QMGR1
it goes to quiescing mdoe and just remains in that mode and at that point have to issue
endmqm -i QMGR1
Yes, that is not a recommended practice but that is what we have to resort to.
From my understanding the queue manager will go into quiescing if
the application connected is waiting for qmgr to finish the work or if connections to teh queue manager have been made without the
FAIL_IF_QUIESCING option
I have checked and connection to the queue manager do use the FAIL_IF_QUIESCING option.
What could be any other reason for the queue manager to be stuck in quiescing? and where could I check?
The logs show only that the queue manager is quiescing.
Thank you!! |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Aug 23, 2016 8:55 am Post subject: Re: Queue manager quiescing |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| hp1411 wrote: |
| I have checked and connection to the queue manager do use the FAIL_IF_QUIESCING option. |
It's not just the connection operations, it's the queue operations as well.
| hp1411 wrote: |
| What could be any other reason for the queue manager to be stuck in quiescing? and where could I check? |
For example, an application has connected with FAIL_IF_QUIESCING but has not specified that on an unlimited wait; perhaps because the application has no restart capability.
Check to see which applications are still connected to the queue manager 10-15 seconds after you issue the original endmqm.
[/quote]
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Tue Aug 23, 2016 9:30 am Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
Presumably when you issue the endmqm all the well behaved applications end but the badly behaved ones stay running. Isn't that a clue about where the problem lies ?
Often in thee cases you have an application which is getting it's input from somewhere else. For example, take an application like AMQSPUT. It can spend it's time sitting in some form of gets() call waiting for user input. The fact that all the MQI calls it issues use the FAIL_IF_QUIESCING flag is irrelevant since when the Queue Manager is trying to come down it is not in an MQI call and therefore doesn't notice. These types of applications should either make periodic checks or sit in a waiting MQGET call (or MQCB) for an indication of QM termination.
Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| hp1411 |
| Posted: Tue Aug 23, 2016 10:13 am Post subject: |
|
|
Newbie
Joined: 17 Oct 2015
Posts: 9
|
Hello Paul And Vitor,
Thank you very much for your replies!
So I take from your replies, I need to check which applications that are causing this mess.
I am not sure how I can check which applications it is that is causing me the headache.
would something like below work? I run this after 10-15 seconds as Vitor recommended and see which channels are still running and then see what applications are connected to those channel?
echo "DISPLAY CHSTATUS(TEST.*) where (status EQ RUNNING))" | runmqsc QMGR1 |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Tue Aug 23, 2016 10:20 am Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
To be honest I'm not sure. After you've issued an endmqm -s that may prevent new applications (ie. RUNMQSC) from running. You may need to do it from an application that is already running.
Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Aug 23, 2016 10:20 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| hp1411 wrote: |
would something like below work? I run this after 10-15 seconds as Vitor recommended and see which channels are still running and then see what applications are connected to those channel?
echo "DISPLAY CHSTATUS(TEST.*) where (status EQ RUNNING))" | runmqsc QMGR1 |
Why are you only checking channels TEST.*? Because:
How do you know it's a "business" application and not (for example) some kind of MQ monitoring or administration tool that's using the queue manager?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| hp1411 |
| Posted: Tue Aug 23, 2016 10:43 am Post subject: |
|
|
Newbie
Joined: 17 Oct 2015
Posts: 9
|
Hello Vitor,
The TEST.* was just an example, I was going to check all channels that apps connected from....but I see your point. I will check all the others as well
Thank you!! |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Aug 23, 2016 10:55 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9396
Location: US: west coast, almost. Otherwise, enroute.
|
Are you assuming that the offending application connects through a channel? What if it's a local app?
You might want to look at DISPLAY CONN command.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| tczielke |
| Posted: Tue Aug 23, 2016 10:57 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA
|
| hp1411 wrote: |
Hello Paul And Vitor,
Thank you very much for your replies!
So I take from your replies, I need to check which applications that are causing this mess.
I am not sure how I can check which applications it is that is causing me the headache.
would something like below work? I run this after 10-15 seconds as Vitor recommended and see which channels are still running and then see what applications are connected to those channel?
echo "DISPLAY CHSTATUS(TEST.*) where (status EQ RUNNING))" | runmqsc QMGR1 |
The following approach of using the -c switch for dspmq is undocumented, but I use it for when I want to see what connections are still active in a quiescing queue manager:
#find all pids who have active connections to a queue manager
for i in `dspmq -c | xargs -n1 | grep PID | cut -d'(' -f2 | cut -d')' -f1`; do ps -p $i -o user,pid,args; done
_________________
Working with MQ since 2010. |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Tue Aug 23, 2016 2:25 pm Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
In my experience it is in fact more likely to be a local application than a client connection. My recollection of the channel code is that when the Queue Manager is trying to end then all channels are ended regardless of what the Application has specified. Certainly well behaved applications will end faster but after a few seconds even badly behaved applications (over a client connection) will be force ended. At least I'm fairly certain that that is how it used to work when I wrote it 
However, local applications aren't disconnected quite so vigorously which led to the need for the dspmq -m QMNAME -c command.
Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| tczielke |
| Posted: Tue Aug 23, 2016 2:51 pm Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA
|
What I don't understand is why the -c switch for dspmq is undocumented. I get that it would take some effort to document some of those undocumented tools like amqrfdm (cluster cache), but how hard is it to document a switch?
"-c = display processes with active connections to the queue manager"
I even asked in a PMR for the -c switch for dspmq to be documented, and I got back a surprising response that this is an unsupported feature and you should avoid relying on it. On z/OS, it is supported to find active connections for a quiescing queue manager, and this is functionality that you sometimes need as a distributed administrator.
_________________
Working with MQ since 2010. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Aug 23, 2016 5:41 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7717
|
| PaulClarke wrote: |
In my experience it is in fact more likely to be a local application than a client connection. My recollection of the channel code is that when the Queue Manager is trying to end then all channels are ended regardless of what the Application has specified. Certainly well behaved applications will end faster but after a few seconds even badly behaved applications (over a client connection) will be force ended. At least I'm fairly certain that that is how it used to work when I wrote it
|
yup, you remember correctly
http://www.ibm.com/support/knowledgecenter/SSFKSJ_7.5.0/com.ibm.mq.ref.adm.doc/q083320_.htm
| Quote: |
-i
Immediate shutdown. The queue manager stops after it has completed all the MQI calls currently being processed. Any MQI requests issued after the command has been issued fail. Any incomplete units of work are rolled back when the queue manager is next started.
Control is returned after the queue manager has ended.
The effect on any client applications connected through a server-connection channel is equivalent to a STOP CHANNEL command issued in FORCE mode.
|
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
|
|
