| Author |
Message
|
| PeterPotkay |
 Posted: Wed Dec 31, 2003 10:39 am Post subject: Message Driven Beans and FAIL_IF_QUIESCING |
 |
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
How to Message Driven Beans (MDBs) and the FAIL_IF_QUIESCING option relate?
The reason I ask is that I was helping an application run through some testing. One of the things I did was bring down the Queue Manager (endmqm -i QUEUEMANAGER). The QM came down with no problems.
But when I tried to start the QM, it wouldn't. It kept failing:
| Code: |
$ strmqm QUEUEMANAGER
AMQ8041: The queue manager cannot be restarted or deleted because processes,
that were previously connected, are still running.
Process 26223 is still running.
Process 20837 is still running.
AMQ7018: The queue manager operation cannot be completed.
|
Turns out these 2 processes were the MDBs! The QM would not come up until we killed the MDBs. Why? Is there anyway to configure the MDBs so that they end when the QM comes down? The app people are telling me there is no way to specify this option on the MDB.
Solaris 8
MQ 5.3 CSD04
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
 |
| Prahasith |
| Posted: Wed Dec 31, 2003 11:05 am Post subject: |
|
|
Disciple
Joined: 16 May 2003
Posts: 184
Location: Kansas City
|
| Generally the procedure we follow is bring down the app, before we bring down the queue manager |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Dec 31, 2003 11:06 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
The purpose of bringing down the QM without the app was to simulate the Queue manager crashing unexpectedly.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| bower5932 |
| Posted: Wed Dec 31, 2003 11:39 am Post subject: |
|
|
Jedi Knight
Joined: 27 Aug 2001
Posts: 3023
Location: Dallas, TX, USA
|
| Are you sure the processes are the MDBs? I would have expected the MDBs to be running under the process of the App Server and to have the same process id. Your listing shows two different process id's. What app server are you running? If WebSphere, I'm assuming that you are using full MQ since you are shutting it down rather than ending the embedded JMS via the WAS shutdown. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Dec 31, 2003 2:14 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| bower5932 wrote: |
| I would have expected the MDBs to be running under the process of the App Server and to have the same process id. Your listing shows two different process id's. |
He might have two instances of the EJB container running on the same box.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Dec 31, 2003 3:07 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Jeff is correct. I remember during this testing that they killed the first instance, and then asked me to try and restart the QM again. This time I got the same error message, but only 1 pid was listed. Then they killed the second instance, and the QM came up 100% the next time I tried.
I emailed them to ask exactly what App Server they are using. I *think* it is WebLogics, but I am not sure. Looks like we'll have to wait until after the holiday to get the answer from them.
I also asked them if they set the FAILIFQUIESCE option on the Queue Connection Factory object as well as the Queue object. I am assuming the MDB uses these 2 objects, and thus if the option is set there, the MDB would respect it? But everything I have read about MDBs never mentions this option, or the MDB ending if there is an underlying problem. From what I have read, MDBs are these little indestructable tanks that keep trying to read messages no matter what!
It seems weird that the QM had a problem starting back up. I would have thought processes still tied into a QM would have made it difficult to shut down. And if the QM was shutdown (it was), then how can any processess still be connected???
Oh, and does a Queue object in JMS support the FAIL_IF_QUIESCE option? I don't see it anywhere in the Java Manual, but the JMS Admin GUI support pack allows me to create a Queue object while choosing yes or no for FAIL_IF_QUIESCING. So I am guessing yes?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Wed Dec 31, 2003 3:41 pm Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
| Quote: |
| It seems weird that the QM had a problem starting back up. I would have thought processes still tied into a QM would have made it difficult to shut down. And if the QM was shutdown (it was), then how can any processess still be connected??? |
Hi Peter,
don't know if this helps in your case, but upon (some) shutdowns we also kill the shared memory which can be shown using: ipcs -a | grep mq and then kill them using: ipcrm -s or -m if it is a semaphore or memory.
If there is any locked stuff between MQ and the App this will shut it down...
Michael
*** just checked if I could find the post that mentions the 'clean' way of killing the shared memory lock ***
http://www.mqseries.net/phpBB2/viewtopic.php?t=264&highlight=amqiclen
Last edited by Michael Dag on Mon Jan 05, 2004 6:58 am; edited 1 time in total |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Dec 31, 2003 4:19 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
At least in WAS, you can cycle the MDBs separate from the whole AppServer/EJB Container.
That might help, although it would likely be a manual process.
Also, how long did you wait between shutdown and startup of the QM? Was it just a quick cycle?
There may be some kinds of timeouts on the MDBs in effect - like waiting to notice that the channel has closed, or waiting to notice that the QM is quiescing or etc, or waiting for the EJBContainer to pass cycles to the MDB listener, to pass cycles to the MDBs themselves....
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Jan 05, 2004 6:37 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Jeff, I waited about 45 minutes before trying to restart the QM.
Do MDBs not allow this? Will I have to insure that all MDBs are ended before I bring down the QM, or if the QM ends before the MDBs come down, then I have to kill them before trying to bring up the QM? 
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Jan 05, 2004 7:55 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| PeterPotkay wrote: |
| Jeff, I waited about 45 minutes before trying to restart the QM. |
Well, it's not a timeout issue then. Unless you've got some really long idle times on your channels. 
| PeterPotkay wrote: |
| Do MDBs not allow this? Will I have to insure that all MDBs are ended before I bring down the QM, or if the QM ends before the MDBs come down, then I have to kill them before trying to bring up the QM? |
The MDB listener port, at least, should notice that the QM is down, give up it's connection, and go into some sort of retrying state. At least, I would expect it to - but I've been wrong on this kind of thing before... 
Have you found out yet what App Server/JMS provider your client is using? Because there may be some sort of configuration for the MDBs /MDB Listener port that controls this.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Jan 05, 2004 8:15 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
The app server is Weblogic 7.0 sp4.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Jan 20, 2004 12:40 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
On a Unix system, when a queue manager starts it acquires shared memory and
semaphore resources. It uses these during normal operation. If you then end
the queue manager with "endmqm QMGRNAME" then all of these resources will
get cleaned up and there will be no MQ artifacts left in memory (semaphores
and shared memory segments). On the other hand if you issue endmqm with the
-i parameter (as I did during our test) or the -p parameter then these
shared memory and semaphore resources will NOT get cleaned up. That means
that the queue manager will end but the resources it used in memory will
not get cleaned up. This will happen whether my MDBs are connected or
not, which is why it doesn't have anything to do really with MDBs
specifically.
The problem with this is that when you restart the queue manager it will
attempt to acquire new shared memory and semaphore resources, as it
normally does. However, if OLD resources are still resident in memory
(because they weren't cleaned up during the last MQ shutdown) then it is
possible that the current instance of the queue manager (that is starting)
might interpret these resources as belonging to processes that are still
running. This is why we got the error messages we saw.
In most cases the queue manager will start OK even if it finds old
resources in memory, but it can encounter the error we saw.
The -i switch on the endmqm command means shut down immediately. Do not
wait for connected programs to end when they want to (like MDBs). Without
that -i switch, we run the risk of our QM shutdown waiting forever for the
MDBs to end. This is why I always use the -i switch when ending a QM.
At 5.3, there is the undocumented amqiclen command that cleans up these
resources.
We are testing this command at the end of our shutdown scripts and at the
beginning
of our startup scripts. It cleans up old memory resources if they are
present
after a QM has come down, eliminating the problem we saw on subsequent
startups.
Back to MDBs:
I still think we will research the polling interval setting to help MDBs
ralize that the QM is gone, as that can
only help prevent problems.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
|
|
