Author |
Message
|
suneel |
Posted: Thu Mar 03, 2011 2:34 am Post subject: reason '2009' ('MQRC_CONNECTION_BROKEN'). |
|
|
Voyager
Joined: 06 Nov 2009 Posts: 89
|
Hi,
When application is trying to connect MQ server application is getting reason '2009' ('MQRC_CONNECTION_BROKEN'). But i could see QM and listeners are running fine. Even though application is bounced still application is getting same exception. I found below FDC from MQ errors.
This issue is occurring very frequently. Whenever it occurs we had go go for QM bounce. Please provide a permanent solution.
+-----------------------------------------------------------------------------+
| |
| WebSphere MQ First Failure Symptom Report |
| ========================================= |
| |
| Date/Time :- Wed March 02 2011 13:51:49 GMT |
| UTC Time :- 1299073909.041680 |
| UTC Time Offset :- 0 (GMT) |
| Host Name :- a01pp02 |
| Operating System :- Linux 2.6.18-128.el5 |
| PIDS :- 5724H7224 |
| LVLS :- 7.0.1.3 |
| Product Long Name :- WebSphere MQ for Linux (x86 platform) |
| Vendor :- IBM |
| Probe Id :- XC338001 |
| Application Name :- MQM |
| Component :- xehAsySignalHandler |
| SCCS Info :- lib/cs/unix/amqxerrx.c, 1.242.1.2 |
| Line Number :- 718 |
| Build Date :- Aug 12 2010 |
| CMVC level :- p701-103-100813 |
| Build Type :- IKAP - (Production) |
| Effective UserID :- 1000 (mqm) |
| Real UserID :- 1000 (mqm) |
| Program Name :- rsvmqtrn |
| Addressing mode :- 32-bit |
| Process :- 28795 |
| Process(Thread) :- 28797 |
| Thread :- 2 |
| ThreadingModel :- PosixThreads |
| UserApp :- FALSE |
| Last HQC :- 0.0.0-0 |
| Last HSHMEMB :- 0.0.0-0 |
| Major Errorcode :- xecE_W_UNEXPECTED_ASYNC_SIGNAL |
| Minor Errorcode :- OK |
| Probe Type :- MSGAMQ6209 |
| Probe Severity :- 3 |
| Probe Description :- AMQ6209: An unexpected asynchronous signal (2 : |
| SIGINT) has been received and ignored. |
| FDCSequenceNumber :- 0 |
| Arith1 :- 2 (0x2) |
| Comment1 :- SIGINT |
| Comment2 :- Signal sent by pid 0 |
| |
+-----------------------------------------------------------------------------+ |
|
Back to top |
|
 |
mqjeff |
Posted: Thu Mar 03, 2011 4:46 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008 Posts: 17447
|
That FDC doesn't say anything about the source of the problem.
2009 is either an issue with too many channels being open - perhaps the app is forgetting to close them or is opening and closing them too often - or it's an issue with the network. |
|
Back to top |
|
 |
Mr Butcher |
Posted: Thu Mar 03, 2011 5:01 am Post subject: |
|
|
 Padawan
Joined: 23 May 2005 Posts: 1716
|
Quote: |
Whenever it occurs we had go go for QM bounce |
this is very suspicious and i think mqjeff already pointed you into the correct direction to check if some of your ressources are exhausted. what about the amqerrxx.log files? no entries in there?
Quote: |
Please provide a permanent solution. |
many different issues may pop up with 2009, so i am afraid you need to uninstal for a permanent solution  _________________ Regards, Butcher |
|
Back to top |
|
 |
gbaddeley |
Posted: Thu Mar 03, 2011 5:58 pm Post subject: |
|
|
 Jedi Knight
Joined: 25 Mar 2003 Posts: 2538 Location: Melbourne, Australia
|
2009 covers a multitude of possible networking and app process related issues, none of which MQ has control over or can effectively recover. It generally indicates an establish comms session (eg. a TCP/IP Socket) between two Message Channel Agents (I include the MQ Client libraries here) failed unexpectedly. Sometimes there is an errno, winsock or zOS oesocket return code in the MQ logs which may shed some light on the real issue. _________________ Glenn |
|
Back to top |
|
 |
santnmq |
Posted: Thu Mar 03, 2011 9:19 pm Post subject: |
|
|
Centurion
Joined: 11 Jan 2011 Posts: 125
|
connection to the server can be broken beacuse of the several reasons including network issues also. Make sure that at any point in time the total number of client connections are below the specified limit.
Also, if your client connection breaks because of the error in the client program, the connection can not be established again till you restart the client app and the queue manager.
Thoroughly monitor the client connections and see what is the exact cause of this issue. |
|
Back to top |
|
 |
bruce2359 |
Posted: Thu Mar 03, 2011 10:00 pm Post subject: |
|
|
 Poobah
Joined: 05 Jan 2008 Posts: 9469 Location: US: west coast, almost. Otherwise, enroute.
|
Quote: |
till you restart the client app and the queue manager. |
Really? _________________ I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
Back to top |
|
 |
santnmq |
Posted: Fri Mar 04, 2011 3:08 am Post subject: |
|
|
Centurion
Joined: 11 Jan 2011 Posts: 125
|
I had faced the similar kind of problem which gets resolved once you restart the client app or the queue manager.
What I could probably makeout from that is, it was necessary in that scenario to first close all the previous connections to get a new successful connection to the server.
It may not be true for all scenarios. |
|
Back to top |
|
 |
bruce2359 |
Posted: Fri Mar 04, 2011 6:38 am Post subject: |
|
|
 Poobah
Joined: 05 Jan 2008 Posts: 9469 Location: US: west coast, almost. Otherwise, enroute.
|
Quote: |
...gets resolved once you restart the client app or the queue manager. |
Restarting the qmgr is like rebooting Windoze - it doesn't really fix the underlying cause of the problem, or anything else. _________________ I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
Back to top |
|
 |
suneel |
Posted: Tue Mar 08, 2011 1:51 am Post subject: |
|
|
Voyager
Joined: 06 Nov 2009 Posts: 89
|
Hi,
Restarting the QM is not a permanent solution. Eventhough Applications are bounced still we are facing the same issue |
|
Back to top |
|
 |
exerk |
Posted: Tue Mar 08, 2011 4:06 am Post subject: |
|
|
 Jedi Council
Joined: 02 Nov 2006 Posts: 6339
|
suneel wrote: |
Restarting the QM is not a permanent solution. Even though Applications are bounced still we are facing the same issue |
You can bounce your applications as much as you like but if the problem is at the queue manager end it makes no difference. As has been suggested, check the queue manager error logs for MaxChannels/MaxActiveChannels events. _________________ It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
Back to top |
|
 |
vmcgloin |
Posted: Tue Mar 08, 2011 4:50 am Post subject: |
|
|
Knight
Joined: 04 Apr 2002 Posts: 560 Location: Scotland
|
Clearly you need to examine the qmgr logs as already suggested given that restarting the apps does not help. Does it really not help, or it is that the problem occurs again some time later?
Do you use keepalive? Is there something in your network breaking connections? Do you have code in your applications to reconnect cleanly when getting a 2009?
On your FFST (unrelated to the 2009 as has already been stated) you should read http://hursleyonwmq.wordpress.com/2007/05/04/introduction-to-ffsts/ (an old doc I still regularly refer new admin's to). |
|
Back to top |
|
 |
|