| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| MQ Commands Hang? |
« View previous topic :: View next topic » |
| Author |
Message
|
| GLL0905 |
 Posted: Wed Aug 08, 2001 8:21 am Post subject: |
 |
|
Apprentice
Joined: 25 Jun 2001
Posts: 40
Location: gauteng, sa
|
Hi, Sometimes when i issue mq commands(runmqsc, endmqm, dis chs/q) on solaris(mq 5.1) the commands just freezes until i 'Crtl C'. I am not sure if this is related to the os or to mq itself. But when i issue a top command i don't see any mq related processes running although users are timing out on their mq triggered applications. I don't get any FDC files or errors log files. My unix administrator claims he can't find anything
either. Recently i got the same problem on NT. The only solution i have is to issue a kill -9 -1 as mq user and start the qmanager again. Any ideas? 
Thanks, gll
graham.lekota@liberty.co.za |
|
| Back to top |
|
 |
| bduncan |
| Posted: Wed Aug 08, 2001 12:02 pm Post subject: |
|
|
Padawan
Joined: 11 Apr 2001
Posts: 1554
Location: Silicon Valley
|
I have dealt with this before. Usually it is a symptom of the queue manager being in dire straits, and I have seen this happen for several reasons. Here is a list off the top of my head, there is probably many more, but you'll want to see if your situation fits any of these:
1. /var/mqm/ is full. If this partition hosting /var/mqm fills up, mqm is unable to fufill any commands. After killing the queue manager and starting it again everything will go back to normal because if the file system was full of non-persistent messages, then they will go away when the queue manager is recycled. Not many people realize that even non-persistent messages cause the size of the "q" files for each queue to increase, and it only goes back down after the messages have been removed or the queue manager is recycled.
2. You are doing really big units of work and you are exceeding the limit for the number of messages within a single unit of work. When this occurs, the queue manager pretty much goes belly up, but if you try to go into runmqsc, you will find that it is non-responsive.
3. Some essential MQ processes have been killed (possibly by you). I noticed that you said you had to issue "kill -9" on the MQ processes. Although you did this AFTER it began exhibiting the weird symptoms, doing such things can actually cause problems like what you are seeing. So my advice is even if the queue manager is non-responsive don't resort to kill -9's unless all else fails. MQSeries actually keeps an internal track of what processes it has spawned (it doesn't depend on the OS to keep track of this) and sometimes if you issue a kill -9, MQSeries will be unaware of this and think the process is still running. This will cause you quite a bit of trouble when you get around to starting the queue manager again, because MQSeries will have no record of "it" killing these processes (because "you" did) and it will look for these processes, which won't be there, and you will get all sorts of weird errors saying that "essential MQSeries process PID# could not be found". So I would first try to end the queue manager using:
endmqm
then:
endmqm -i
and finally:
endmqm -p
if none of these work, you can resort to using kill, but there is actually a document IBM put out (can't recall where it is right now) that lists the order in which you should kill MQ processes if you are forced to do so. I would highly recommend finding this article (anyone know?) and following it exactly, otherwise you'll have lot's of problems even after you restart the queue manager...
_________________
Brandon Duncan
IBM Certified MQSeries Specialist
MQSeries.net forum moderator |
|
| Back to top |
|
|
| bduncan |
| Posted: Wed Aug 08, 2001 3:56 pm Post subject: |
|
|
Padawan
Joined: 11 Apr 2001
Posts: 1554
Location: Silicon Valley
|
Ahh, I found that reference on how to shut down a queue manager manually. If you have to resort to using "kill" here's how you minimize the damage:
Taken from MQSeries Administrator Manual, page 359
Stopping a queue manager manually
The standard way of stopping queue managers, using the endmqm command, should work even in the event of failures within the queue manager. In exceptional circumstances, if this method of stopping a queue manager fails, you can use one of the procedures described here to stop it manually.
Stopping queue managers in MQSeries for UNIX systems
To stop a queue manager running under MQSeries for UNIX systems:
1. Find the process IDs of the queue manager programs that are still running using the ps command. For example, if the queue manager is called QMNAME, the following command can be used:
ps -ef | grep QMNAME
2. End any queue manager processes that are still running. Use the kill command, specifying the process IDs discovered using the ps command.
Note: Processes that fail to stop can be ended using kill -9.
End the processes in the following order:
amqpcsea - command server
amqhasmx - logger
amqharmx - log formatter (LINEAR logs only)
amqzllp0 - checkpoint processor
amqzlaa0 - queue manager agents
amqzxma0 - processing controller
amqrrmfa - Repository process (for clusters)
Note: Manual stopping of the queue manager may result in FFSTs being taken, and the production of FDC files in /var/mqm/errors. This should not be regarded as a defect in the queue manager. The queue manager should restart normally, even after having been stopped using this method.
Attention!
If you do not shut down a queue manager properly, you run the risk of MQSeries not tidying up operating system resources such as semaphores and shared memory sets. This can result in a gradual degradation of system performance and in you having to reboot your system.
Stopping queue managers in MQSeries for Windows NT
To stop a queue manager running under MQSeries for Windows NT:
1. List the names (IDs) of the processes currently running using the Windows NT Process Viewer (PView)
2. Stop the processes using PView in the following order (if they are running):
AMQPCSEA.EXE - The command server
AMQHASMN.EXE - The logger
AMQHARMN.EXE - Log formatter (LINEAR logs only)
AMQZLLP0.EXE - Checkpoint process
AMQZLAA0.EXE - LQM agents
AMQZTRCN.EXE - Trace
AMQZXMA0.EXE - Execution controller
AMQXSSVN.EXE - Shared memory servers
AMQRRMFA.EXE - The repository process
(for clusters)
3. Stop the queue manager service using the Windows NT Control Panel.
4. If you have tried all methods and the queue manager has not stopped, reboot
your system.
_________________
Brandon Duncan
IBM Certified MQSeries Specialist
MQSeries.net forum moderator
[ This Message was edited by: bduncan on 2001-08-08 16:57 ] |
|
| Back to top |
|
|
| GLL0905 |
| Posted: Fri Aug 10, 2001 4:57 am Post subject: |
|
|
Apprentice
Joined: 25 Jun 2001
Posts: 40
Location: gauteng, sa
|
Thanks for the ideas. I think the problem(on solaris) could be related to reasons 1 or 2.
MQSeries is used heavily on the two servers the problem occured. Normally in the morning
before 0700 hours several triggered queues are piled up with messages because trigger monitors are brought up once batch runs and backups are finished.
I will monitor the system(disk space, etc) and hopefully when the problem occurs(if it does) i will pick up any abnormalities that might be attributed to the problem.
ta.
|
|
| Back to top |
|
|
| olan022 |
| Posted: Wed Jul 23, 2003 5:58 am Post subject: |
|
|
Acolyte
Joined: 17 Apr 2002
Posts: 50
|
bduncan,
I'm interested in cause #2 above - just a quick question, when you say the queue manager goes belly up - what actually happens? Does it appear to hang but all the processes are still running and you are forced to stop and start the qmanager? |
|
| Back to top |
|
|
| bduncan |
| Posted: Wed Jul 23, 2003 12:42 pm Post subject: |
|
|
Padawan
Joined: 11 Apr 2001
Posts: 1554
Location: Silicon Valley
|
Looking at my post (from almost 2 years ago!) I think I was incorrect. When the unit of work exceeds a certain limit, the next operation under that unit of work returns with a special reason code saying the UOW is too big. At the same time, the queue manager will automatically roll back the entire UOW. Since the UOW must be really big to cause this problem, the rollback takes quite a while (especially if you are using XA). This can cause the queue manager's response time to slow way down, but I don't think it hangs indefinitely. When the rollback completes, everything should be back to normal.
_________________
Brandon Duncan
IBM Certified MQSeries Specialist
MQSeries.net forum moderator |
|
| Back to top |
|
|
| queuetip |
| Posted: Thu Mar 23, 2006 1:02 pm Post subject: Re: MQ Commands |
|
|
Acolyte
Joined: 03 Feb 2005
Posts: 67
|
| Quote: |
<b>Stopping a queue manager manually</b>
The standard way of stopping queue managers, using the endmqm command, should work even in the event of failures within the queue manager. In exceptional circumstances, if this method of stopping a queue manager fails, you can use one of the procedures described here to stop it manually.
<b>Stopping queue managers in MQSeries for Windows NT</b>
To stop a queue manager running under MQSeries for Windows NT:
1. List the names (IDs) of the processes currently running using the Windows NT Process Viewer (PView)
2. Stop the processes using PView in the following order (if they are running):
<b>AMQPCSEA.EXE</b> - The command server
<b>AMQHASMN.EXE</b> - The logger
<b>AMQHARMN.EXE</b> - Log formatter (LINEAR logs only)
<b>AMQZLLP0.EXE</b> - Checkpoint process
<b>AMQZLAA0.EXE</b> - LQM agents
<b>AMQZTRCN.EXE</b> - Trace
<b>AMQZXMA0.EXE</b> - Execution controller
<b>AMQXSSVN.EXE</b> - Shared memory servers
<b>AMQRRMFA.EXE</b> - The repository process
(for clusters)
3. Stop the queue manager service using the Windows NT Control Panel.
4. If you have tried all methods and the queue manager has not stopped, reboot
your system. |
When I look at the sequence above for the shut down, I do not see any reference to shutdown the listener (runmqlsr.exe) not channel initiator (runmqchi.exe). Should they be included somewhere in the sequence above?
If not, how do they terminate?
Thanks,
Mike |
|
| Back to top |
|
|
| vennela |
| Posted: Thu Mar 23, 2006 1:16 pm Post subject: |
|
|
Jedi Knight
Joined: 11 Aug 2002
Posts: 4055
Location: Hyderabad, India
|
| I would do it at the end, after verifying QMGR is down, but that's just me. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Mar 23, 2006 1:18 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
The chin should shut down on it's own, I think.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Mar 24, 2006 7:41 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
