| Author |
Message
|
| rein8 |
 Posted: Wed Oct 10, 2007 12:05 am Post subject: Connection between WAS and MQ lost |
 |
|
Newbie
Joined: 18 Apr 2007
Posts: 8
Location: Schiphol Airport
|
After a redeployment of an application in WAS v5 the application can't read from the queue's anymore. I've got MQ v5.3. I stop WAS, uninstall, install, start WAS application server again.
The MQ queue connection factory of WAS connects with inter-process binding to MQ.
After a restart of MQ the application connects again.
Can I reconfigure WAS or MQ so that I don't have to restart MQ for a deployment in WAS? |
|
| Back to top |
|
 |
| mvic |
| Posted: Wed Oct 10, 2007 12:34 am Post subject: Re: Connection between WAS and MQ lost |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| rein8 wrote: |
| Can I reconfigure WAS or MQ so that I don't have to restart MQ for a deployment in WAS? |
Not sure you needed to restart MQ. What were the error messages? Any FDC files in /var/mqm/errors ? |
|
| Back to top |
|
|
| rein8 |
| Posted: Wed Oct 10, 2007 1:29 am Post subject: |
|
|
Newbie
Joined: 18 Apr 2007
Posts: 8
Location: Schiphol Airport
|
I have no FDC reports related to the problem of not being able to read from the queue's.
We had this problem before. I stopped MQ. MQ went to state quiescing, then stopped, no mq processes were running anymore, when I tried to start MQ I got the message that several processes that were previously connected still exist, with ipcs I saw that several shared memory segments still existed, then I stopped WAS app servers, restarted MQ, and started WAS, then everything was fine again.
I do see:
AMQ7469: Transactions rolled back to release log space.
in the queue manager error log file. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Oct 10, 2007 1:33 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| rein8 wrote: |
AMQ7469: Transactions rolled back to release log space.
in the queue manager error log file. |
Have you tried increasing the available logs?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mvic |
| Posted: Wed Oct 10, 2007 1:43 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| rein8 wrote: |
| We had this problem before. I stopped MQ. |
Why was the decision made to restart the queue manager?
| Quote: |
| MQ went to state quiescing, then stopped, no mq processes were running anymore, when I tried to start MQ I got the message that several processes that were previously connected still exist, with ipcs I saw that several shared memory segments still existed, then I stopped WAS app servers, restarted MQ, and started WAS, then everything was fine again. |
Lots of people run client connections (ie. over TCP/IP) from their App Server to MQ, instead of local bindings. Probably (?) this is the reason. |
|
| Back to top |
|
|
| rein8 |
| Posted: Wed Oct 10, 2007 1:57 am Post subject: |
|
|
Newbie
Joined: 18 Apr 2007
Posts: 8
Location: Schiphol Airport
|
Yes, I think that's the reason we had to bring the appserver down as well.
The decision to restart MQ was made because messages were piling up in MQ and had to be read by the application. From previous experience we knew that restarting MQ and WAS would solve the problem. But I don't want to restart MQ every time we're doing a new deployment ... |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Oct 10, 2007 2:07 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Make sure MQ is running at CSD13. Make sure your WAS is running at the most recent fix level of WAS v5.0.
Start building a new system that will run a supported version of MQ and a supported version of WAS.
You need to do more analysis to determine why the app stops being able to read messages. It's not a good sign that MQ is rolling back transactions to conserve log space, but it may be completely irrelevant. Or it might be causing your message listeners to stop.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| rein8 |
| Posted: Wed Oct 10, 2007 3:13 am Post subject: |
|
|
Newbie
Joined: 18 Apr 2007
Posts: 8
Location: Schiphol Airport
|
We have CSD7 and WAS v5.1.1.9
A developer suggested to increase the number of active channels. Would this be a possible solution until we've upgraded? |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Oct 10, 2007 3:16 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| jefflowrey wrote: |
| You need to do more analysis to determine why the app stops being able to read messages. |
Most system administrators don't consider "asking a developer" to be analysis. Look ath the errors in the QM error log, look at the errors in the WAS log. Figure out, as best you can, what is going wrong FIRST.
It may be that increasing MaxActiveChannels will solve the problem. It may not be. It may be that increasing MaxActiveChannels will cause MORE problems, because the channels use up necessary system resources and everything falls over.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| mvic |
| Posted: Wed Oct 10, 2007 3:24 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| jefflowrey wrote: |
| It may be that increasing MaxActiveChannels will solve the problem. |
Since they are using server bindings (I think that's a fair interpretation of what was said) to connect from the app server, altering channel settings would not affect these connections.
One thing I should have emphasized is that client connections are over TCP/IP, and hence should decouple the app from the queue manager, meaning the queue manager can be restarted (still don't know why this was needed, by the way) without forcing the app server to need to be restarted too. |
|
| Back to top |
|
|
| rein8 |
| Posted: Wed Oct 10, 2007 3:39 am Post subject: |
|
|
Newbie
Joined: 18 Apr 2007
Posts: 8
Location: Schiphol Airport
|
From previous similar experiences on our test server we noticed that after restarting the environment the application could read from the queue's again and we didn't have time to figure out what was happening.
I also don't know why we had to restart but I do know that it worked. My guess is that it had to do with the WAS processes having a lock on shared memory.
From what I understand from the answers:
1/ upgrade to v6
2/ change connection from bindings to tcp/ip |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Oct 10, 2007 4:11 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
NO.
The Answer is "STOP GUESSING".
Start analyzing.
An upgrade make cause the problem to disappear, it might not. Either way, it should be done. And either way you need to FIGURE OUT what the problem is.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| rein8 |
| Posted: Wed Oct 10, 2007 6:19 am Post subject: |
|
|
Newbie
Joined: 18 Apr 2007
Posts: 8
Location: Schiphol Airport
|
| I'm looking for possible causes so I know where to look. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Oct 10, 2007 6:44 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| jefflowrey wrote: |
| Look at the errors in the QM error log, look at the errors in the WAS log. Figure out, as best you can, what is going wrong FIRST. |
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Oct 10, 2007 3:37 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
From what you described I'll let you in on a little secret.
We use MQ 5.3 with WAS 5.x too, but are in the process of upgrading to MQ 6.0
In WAS 5.x we were never able to recycle the qmgr while WAS was running MDBs (bindings connection). The qmgr is left hanging some processes related to WAS.
So if we had to recycle the qmgr: 2 possibilities
Do it during the offline backup window (a must for a good offline backup)
Do it after the MDB service was stopped / WAS was stopped.
Restart qmgr & restart WAS...
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
