| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Could Client Channels be the issue? |
« View previous topic :: View next topic » |
| Author |
Message
|
| sandy_harry |
 Posted: Mon Apr 03, 2006 12:00 pm Post subject: Could Client Channels be the issue? |
 |
|
Newbie
Joined: 29 Jun 2005
Posts: 8
Location: Portsmouth, NH
|
Hi,
I have started this new thread on request. The problem I have is similar to the one reporterd in http://www.mqseries.net/phpBB2/viewtopic.php?p=130902#130902. Here is my story:
I have 2 MQ client channels defined to 2 different QManagers from my WAS 5.0 node. Listener ports are defined on the WAS nodes for the 2 queues on these remote QManagers. The app servers restart at 3:00 am every morning, but at times there is no message sent from the 2 remote QManagers till about 11:30 am. When the first message arrives, I get the following exception trace in my SystemOut.Logs:
[4/3/06 11:26:13:700 EDT] 21e32ac2 JMSExceptionL E WMSG0018E: Error on JMSConnection for MDB CCOFeedMDB , JMSDestination jms/CCORequestQ : javax.jms.JMSException: MQJMS2002: failed to get message from MQ queue
at com.ibm.mq.jms.services.ConfigEnvironment.newException(ConfigEnvironment.java(Inlined Compiled Code))
at com.ibm.mq.jms.MQSession.consume(MQSession.java(Compiled Code))
at com.ibm.mq.jms.MQSession.run(MQSession.java(Compiled Code))
at com.ibm.ejs.jms.JMSSessionHandle.run(JMSSessionHandle.java:923)
at com.ibm.ejs.jms.listener.ServerSession.connectionConsumerOnMessage(ServerSession.java:697)
at com.ibm.ejs.jms.listener.ServerSession.onMessage(ServerSession.java:482)
at com.ibm.ejs.jms.listener.ServerSession.dispatch(ServerSession.java:449)
at java.lang.reflect.Method.invoke(Native Method)
at com.ibm.ejs.jms.listener.ServerSessionDispatcher.dispatch(ServerSessionDispatcher.java:37)
at com.ibm.ejs.container.MDBWrapper.onMessage(MDBWrapper.java:91)
at com.ibm.ejs.container.MDBWrapper.onMessage(MDBWrapper.java:127)
at com.ibm.ejs.jms.listener.ServerSession.run(ServerSession.java:372)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:672)
---- Begin backtrace for Nested Throwables
com.ibm.mq.MQException: MQJE001: Completion Code 2, Reason 2009
at com.ibm.mq.jms.MQSession.consume(MQSession.java(Compiled Code))
at com.ibm.mq.jms.MQSession.run(MQSession.java(Compiled Code))
at com.ibm.ejs.jms.JMSSessionHandle.run(JMSSessionHandle.java:923)
at com.ibm.ejs.jms.listener.ServerSession.connectionConsumerOnMessage(ServerSession.java:697)
at com.ibm.ejs.jms.listener.ServerSession.onMessage(ServerSession.java:482)
at com.ibm.ejs.jms.listener.ServerSession.dispatch(ServerSession.java:449)
at java.lang.reflect.Method.invoke(Native Method)
at com.ibm.ejs.jms.listener.ServerSessionDispatcher.dispatch(ServerSessionDispatcher.java:37)
at com.ibm.ejs.container.MDBWrapper.onMessage(MDBWrapper.java:91)
at com.ibm.ejs.container.MDBWrapper.onMessage(MDBWrapper.java:127)
at com.ibm.ejs.jms.listener.ServerSession.run(ServerSession.java:372)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:672)
[4/3/06 11:26:13:723 EDT] 21e32ac2 JMSExceptionL E WMSG0057E: Error on JMSConnection for MDB CCOFeedMDB , JMSDestination jms/CCORequestQ , JMS Linked Exception : com.ibm.mq.MQException: MQJE001: Completion Code 2, Reason 2009
at com.ibm.mq.jms.MQSession.consume(MQSession.java(Compiled Code))
at com.ibm.mq.jms.MQSession.run(MQSession.java(Compiled Code))
at com.ibm.ejs.jms.JMSSessionHandle.run(JMSSessionHandle.java:923)
at com.ibm.ejs.jms.listener.ServerSession.connectionConsumerOnMessage(ServerSession.java:697)
at com.ibm.ejs.jms.listener.ServerSession.onMessage(ServerSession.java:482)
at com.ibm.ejs.jms.listener.ServerSession.dispatch(ServerSession.java:449)
at java.lang.reflect.Method.invoke(Native Method)
at com.ibm.ejs.jms.listener.ServerSessionDispatcher.dispatch(ServerSessionDispatcher.java:37)
at com.ibm.ejs.container.MDBWrapper.onMessage(MDBWrapper.java:91)
at com.ibm.ejs.container.MDBWrapper.onMessage(MDBWrapper.java:127)
at com.ibm.ejs.jms.listener.ServerSession.run(ServerSession.java:372)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:672)
[4/3/06 11:26:13:724 EDT] 21e32ac2 MDBListenerIm I WMSG0043I: MDB Listener lsnrCCOFeedMQ stopped for JMSDestination jms/CCORequestQ
[4/3/06 11:26:15:690 EDT] 21e32ac2 MDBListenerIm I WMSG0058I: Listener Port lsnrCCOFeedMQ will attempt to restart in 60 seconds
[4/3/06 11:26:15:691 EDT] 21e32ac2 ConnectionEve A J2CA0056I: The Connection Manager received a fatal connection error from the Resource Adaptor for resource JMS$RRCF_client. The exception which was received is javax.jms.JMSException: MQJMS2002: failed to get message from MQ queue
[4/3/06 11:27:15:735 EDT] 7763eaa5 MDBListenerIm I WMSG0042I: MDB Listener lsnrCCOFeedMQ started successfully for JMSDestination jms/CCORequestQ
[4/3/06 11:27:15:737 EDT] 51312b43 JMSExceptionL E WMSG0018E: Error on JMSConnection for MDB CCOFeedMDB , JMSDestination jms/CCORequestQ : javax.jms.JMSException: MQJMS2002: failed to get message from MQ queue
at com.ibm.mq.jms.services.ConfigEnvironment.newException(ConfigEnvironment.java(Inlined Compiled Code))
at com.ibm.mq.jms.MQSession.consume(MQSession.java(Compiled Code))
at com.ibm.mq.jms.MQSession.run(MQSession.java(Compiled Code))
at com.ibm.ejs.jms.JMSSessionHandle.run(JMSSessionHandle.java:923)
at com.ibm.ejs.jms.listener.ServerSession.connectionConsumerOnMessage(ServerSession.java:697)
at com.ibm.ejs.jms.listener.ServerSession.onMessage(ServerSession.java:482)
at com.ibm.ejs.jms.listener.ServerSession.dispatch(ServerSession.java:449)
at java.lang.reflect.Method.invoke(Native Method)
at com.ibm.ejs.jms.listener.ServerSessionDispatcher.dispatch(ServerSessionDispatcher.java:37)
at com.ibm.ejs.container.MDBWrapper.onMessage(MDBWrapper.java:91)
at com.ibm.ejs.container.MDBWrapper.onMessage(MDBWrapper.java:127)
at com.ibm.ejs.jms.listener.ServerSession.run(ServerSession.java:372)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:672)
---- Begin backtrace for Nested Throwables
com.ibm.mq.MQException: MQJE001: Completion Code 2, Reason 2019
at com.ibm.mq.jms.MQSession.consume(MQSession.java(Compiled Code))
at com.ibm.mq.jms.MQSession.run(MQSession.java(Compiled Code))
at com.ibm.ejs.jms.JMSSessionHandle.run(JMSSessionHandle.java:923)
at com.ibm.ejs.jms.listener.ServerSession.connectionConsumerOnMessage(ServerSession.java:697)
at com.ibm.ejs.jms.listener.ServerSession.onMessage(ServerSession.java:482)
at com.ibm.ejs.jms.listener.ServerSession.dispatch(ServerSession.java:449)
at java.lang.reflect.Method.invoke(Native Method)
at com.ibm.ejs.jms.listener.ServerSessionDispatcher.dispatch(ServerSessionDispatcher.java:37)
at com.ibm.ejs.container.MDBWrapper.onMessage(MDBWrapper.java:91)
at com.ibm.ejs.container.MDBWrapper.onMessage(MDBWrapper.java:127)
at com.ibm.ejs.jms.listener.ServerSession.run(ServerSession.java:372)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:672)
[4/3/06 11:27:15:737 EDT] 51312b43 JMSExceptionL E WMSG0057E: Error on JMSConnection for MDB CCOFeedMDB , JMSDestination jms/CCORequestQ , JMS Linked Exception : com.ibm.mq.MQException: MQJE001: Completion Code 2, Reason 2019
at com.ibm.mq.jms.MQSession.consume(MQSession.java(Compiled Code))
at com.ibm.mq.jms.MQSession.run(MQSession.java(Compiled Code))
at com.ibm.ejs.jms.JMSSessionHandle.run(JMSSessionHandle.java:923)
at com.ibm.ejs.jms.listener.ServerSession.connectionConsumerOnMessage(ServerSession.java:697)
at com.ibm.ejs.jms.listener.ServerSession.onMessage(ServerSession.java:482)
at com.ibm.ejs.jms.listener.ServerSession.dispatch(ServerSession.java:449)
at java.lang.reflect.Method.invoke(Native Method)
at com.ibm.ejs.jms.listener.ServerSessionDispatcher.dispatch(ServerSessionDispatcher.java:37)
at com.ibm.ejs.container.MDBWrapper.onMessage(MDBWrapper.java:91)
at com.ibm.ejs.container.MDBWrapper.onMessage(MDBWrapper.java:127)
at com.ibm.ejs.jms.listener.ServerSession.run(ServerSession.java:372)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:672)
[4/3/06 11:27:15:738 EDT] 51312b43 MDBListenerIm I WMSG0043I: MDB Listener lsnrCCOFeedMQ stopped for JMSDestination jms/CCORequestQ
[4/3/06 11:27:15:771 EDT] 5134ab43 ConnectionEve A J2CA0056I: The Connection Manager received a fatal connection error from the Resource Adaptor for resource JMS$RRCF_client. The exception which was received is javax.jms.JMSException: MQJMS2002: failed to get message from MQ queue
[4/3/06 11:27:20:798 EDT] 5134ab43 WASLogger E CLASSNAME METHODNAME an internal error has occurred. Please contact your system administrator. Detail: com.ibm.mq.MQException: MQJE001: Completion Code 2, Reason 2019
[4/3/06 11:27:20:873 EDT] 51312b43 MDBListenerIm I WMSG0058I: Listener Port lsnrCCOFeedMQ will attempt to restart in 60 seconds
[4/3/06 11:28:20:918 EDT] 19a2aa9f MDBListenerIm I WMSG0042I: MDB Listener lsnrCCOFeedMQ started successfully for JMSDestination jms/CCORequestQ
It does succeed in finally connecting after the second retry, but this is still a concern. Also this is an issue only with Listener ports on Client channels. The referred topic concluded by saying that this was a firewall timeout issue. I need more information to convince my team. Can somebody please help?
If anyone needs any more info from me, please do let me know.
_________________
Regards,
Sandy |
|
| Back to top |
|
 |
| PeterPotkay |
| Posted: Mon Apr 03, 2006 12:15 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
What MQ version are the MQClients? MQ 5.3 CSD8, 9, 10 and 11 all had JMS MQ Client problems, some of which resulted in unexpected 2009 and 2019s, as described in the following link:
http://www-1.ibm.com/support/docview.wss?uid=swg21217437
Oh we had fun with that one....
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Apr 03, 2006 12:18 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| PeterPotkay wrote: |
| unexpected 2009 and 2019s |
Nobody expects the MQRC_CONNECTION_BROKEN!
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Apr 03, 2006 12:23 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
If I tell the apps I am bringing the QM down Saturday night for maintenance, they should expect any of the following:
| Code: |
MQC.MQRC_Q_MGR_NOT_AVAILABLE
MQC.MQRC_Q_MGR_QUIESCING
MQC.MQRC_Q_MGR_STOPPING
MQC.MQRC_CONNECTION_BROKEN
MQC.MQRC_CONNECTION_QUIESCING
MQC.MQRC_CONNECTION_STOPPING
|
The 2009 would be expected if they have an open queue and go to put to it right after the QM came down.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| sandy_harry |
| Posted: Mon Apr 03, 2006 1:09 pm Post subject: |
|
|
Newbie
Joined: 29 Jun 2005
Posts: 8
Location: Portsmouth, NH
|
Sorry.. should have mentioned it earlier..
We have MQ 5.3 on AIX with CSD 12.
_________________
Regards,
Sandy |
|
| Back to top |
|
|
| oz1ccg |
| Posted: Tue Apr 04, 2006 12:26 am Post subject: |
|
|
Yatiri
Joined: 10 Feb 2002
Posts: 628
Location: Denmark
|
Hmm, what is the timeout (inactive time limit) on the firewall ?
Typicly 30 minutes to 60 minutes....
I had som bad expierences with that long time ago, what I did was changeing the TCP/IP keep alive settings on the boxes to get below the Firewall settings.
You have to make sure that the OS Keep Alive parameter is set to a value below a maximum of 1 hour, which is the recommended firewall configuration setting.
You can define this parameter as follows:
On Windows, the Keep Alive setting is a registry key: the KeepAliveTime TCP parameter.
The default value is 2 hours - 7,200,000 milliseconds. For Firewall setting of 1 hour, you will need to change the KeepAliveTime TCP parameter to 1 hour in order to avoid Firewall disconnections due to idle connection timeout.
NOTE: Changes in the registry key settings must be carried out under strict guidance of your System Administrator.
On AIX, the value/frequence used to send keep alive packet can be obtained by using the command:
1. Log in as root.
2. Check the current values:
3. no -o tcp_keepidle
The tcp_keepidle value is the total length of time to keep an idle TCP connection alive.
This is measured in half-second increments.
The default is 14,400 which equals 2 hours
-- Lock it or Lose it -- 
_________________
Regards, Jørgen
Home of BlockIP2, the last free MQ Security exit ver. 3.00
Cert. on WMQ, WBIMB, SWIFT.
Last edited by oz1ccg on Tue Apr 04, 2006 4:53 am; edited 2 times in total |
|
| Back to top |
|
|
| sandy_harry |
| Posted: Tue Apr 04, 2006 4:34 am Post subject: |
|
|
Newbie
Joined: 29 Jun 2005
Posts: 8
Location: Portsmouth, NH
|
Thanks for your help. I'll find out what is the timeout setting on the AIX boxes. Will the connection always fail if the first activity happens after the firewall timeout? I tried the first transaction 4.5 hours after the app servers were restarted and it successfully went through. I dont believe the timeout is more than 5 hours. In that case why did the channel not fail this time?
Also should I change the timeout setting on the app server node box or the remote QM box? or both?
Thanks again!
_________________
Regards,
Sandy |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
