| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Queue manager quiescing |
« View previous topic :: View next topic » |
| Author |
Message
|
| kalam475 |
 Posted: Fri Mar 10, 2017 10:08 am Post subject: Queue manager quiescing |
 |
|
Acolyte
Joined: 16 Jan 2015
Posts: 63
|
Hi team,
We have a situation where my client application running in Jboss server is connecting to my queue managers participating in cluster through CCDT file. My cluster has three qmgrs. two of them hosting the cluster queues(QM1 and QM2) and one qmgr doesn't have any queues it is only for cluster (a full repository). All three queue managers are multi-instance queue manager.
so we wanted to do some failover testing with some load.
System1
QM1(active and full repository)
QM2(standby and half repository)
QM3(standby and half repository)
system 2
QM1(standby and full repository)
QM2(active and half repository)
QM3(active and full repository)
we started the puting the load and i stoped the qmgr QM1 with endmqm -i -s QM1. It is gracefully switchover to system2 and becomes active. meanwhile my application in jboss using CCDT will connect to QM2 and put the message in one of the cluster queue in it.once QM1 is also active in system 2 my application works fine and accepting the load and distributes the load b/w two cluster queues.
The same test we wanted to do it when system 1 shuts down abruptly rather than ending queue manager.While we shutdown the system queue manage is going into quisece state for 2 minute and then switchover to standby during which heavy load comes to JBOSS connection factory. the connection factory assign all its connections to quisece qmgr(because it is the first qmgr in CCDT file) and was not able to close the connections and soon my jboss hangs.
Below is the error:
| Code: |
INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: JMSCC3036:An exception has been delivered to the connection's exception listener: '
[0m[0m11:55:04,803 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: Message : com.ibm.msg.client.jms.DetailedJMSException: JMSWMQ1107: A problem with this connection has occurred.
[0m[0m11:55:04,804 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: An error has occurred with the WebSphere MQ JMS connection.
[0m[0m11:55:04,804 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: Use the linked exception to determine the cause of this error.
[0m[0m11:55:04,804 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: Class : class com.ibm.msg.client.jms.DetailedJMSException
[0m[0m11:55:04,804 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: Stack : com.ibm.msg.client.wmq.common.internal.Reason.reasonToException(Reason.java:595)
[0m[0m11:55:04,804 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:215)
[0m[0m11:55:04,804 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.msg.client.wmq.internal.WMQConnection.consumer(WMQConnection.java:786)
[0m[0m11:55:04,804 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.mq.jmqi.remote.api.RemoteHconn.callEventHandler(RemoteHconn.java:2918)
[0m[0m11:55:04,804 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.mq.jmqi.remote.api.RemoteHconn.driveEventsEH(RemoteHconn.java:629)
[0m[0m11:55:04,804 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.mq.jmqi.remote.impl.RemoteDispatchThread.processHconn(RemoteDispatchThread.java:659)
[0m[0m11:55:04,804 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.mq.jmqi.remote.impl.RemoteDispatchThread.run(RemoteDispatchThread.java:221)
[0m[0m11:55:04,804 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.runTask(WorkQueueItem.java:319)
[0m[0m11:55:04,804 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.msg.client.commonservices.workqueue.SimpleWorkQueueItem.runItem(SimpleWorkQueueItem.java:99)
[0m[0m11:55:04,804 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.run(WorkQueueItem.java:341)
[0m[0m11:55:04,805 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.msg.client.commonservices.workqueue.WorkQueueManager.runWorkQueueItem(WorkQueueManager.java:312)
[0m[0m11:55:04,805 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.msg.client.commonservices.j2se.workqueue.WorkQueueManagerImplementation$ThreadPoolWorker.run(WorkQueueManagerImplementation.java:1227)
[0m[0m11:55:04,805 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: Caused by [1] --> Message : com.ibm.mq.MQException: JMSCMQ0001: WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2009' ('MQRC_CONNECTION_BROKEN').
[0m[0m11:55:04,805 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: Class : class com.ibm.mq.MQException
[0m[0m11:55:04,805 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: Stack : com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:203)
[0m[0m11:55:04,805 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.msg.client.wmq.internal.WMQConnection.consumer(WMQConnection.java:786)
[0m[0m11:55:04,805 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.mq.jmqi.remote.api.RemoteHconn.callEventHandler(RemoteHconn.java:2918)
[0m[0m11:55:04,805 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.mq.jmqi.remote.api.RemoteHconn.driveEventsEH(RemoteHconn.java:629)
[0m[0m11:55:04,805 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.mq.jmqi.remote.impl.RemoteDispatchThread.processHconn(RemoteDispatchThread.java:659)
[0m[0m11:55:04,805 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.mq.jmqi.remote.impl.RemoteDispatchThread.run(RemoteDispatchThread.java:221)
[0m[0m11:55:04,805 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.runTask(WorkQueueItem.java:319)
[0m[0m11:55:04,805 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.msg.client.commonservices.workqueue.SimpleWorkQueueItem.runItem(SimpleWorkQueueItem.java:99)
[0m[0m11:55:04,805 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.run(WorkQueueItem.java:341)
[0m[0m11:55:04,806 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.msg.client.commonservices.workqueue.WorkQueueManager.runWorkQueueItem(WorkQueueManager.java:312)
[0m[0m11:55:04,806 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: : com.ibm.msg.client.commonservices.j2se.workqueue.WorkQueueManagerImplementation$ThreadPoolWorker.run(WorkQueueManagerImplementation.java:1227)
[0m[0m11:55:04,806 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D51434541492E43514D3120202020BF39C258278A2B24]]) wmq.jmsra.rar: '.
[0m[0m11:55:04,806 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (default task-44) wmq.jmsra.rar: JMSCC3036:An exception has been delivered to the connection's exception listener: '
[0m[0m11:55:04,806 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (default task-44) wmq.jmsra.rar: Message : com.ibm.msg.client.jms.DetailedJMSException: JMSWMQ1107: A problem with this connection has occurred.
[0m[0m11:55:04,806 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (default task-44) wmq.jmsra.rar: An error has occurred with the WebSphere MQ JMS connection.
[0m[0m11:55:04,806 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (default task-44) wmq.jmsra.rar: Use the linked exception to determine the cause of this error.
[0m[31m11:55:04,806 ERROR [com.eb.util.GetConnection] (default task-33)
|
So I was thinking to run script that shutsdown to queue managers before system going down and test it. But want to know where the bottle neck is. Is it because the QMGR is in quisece state for too long i think so because i even tried to kill the QMGR process and was switching within 30 seconds where as if i shutdown the system it is taking 2-3 minutes to switchover and is in quisece state.
Thanks in advance. |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Sat Mar 11, 2017 1:34 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Change your connection factory properties to include FAIL_IF_QUIESCING.
This way when JBOSS tries the first queue manager (quiescing) in the CCDT it will move to the next one... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| kalam475 |
| Posted: Sat Mar 11, 2017 3:39 am Post subject: |
|
|
Acolyte
Joined: 16 Jan 2015
Posts: 63
|
If you don't mind me asking how can we set FAIL_IF_QUIESCING in JBOSS 7 standalone-full.xml. I tried to find it out but not able to find
This is my adapter settings:
| Code: |
<subsystem xmlns="urn:jboss:domain:resource-adapters:1.1">
<resource-adapters>
<resource-adapter id="wmq.jmsra.rar">
<archive>
wmq.jmsra.rar
</archive>
<connection-definitions>
<connection-definition class-name="com.ibm.mq.connector.outbound.ManagedConnectionFactoryImpl" jndi-name="MQDEVJNDI" pool-name="MQ.CONNECTIONFACTORY.NAME">
<config-property name="ccdtURL">
file://opt/MQ/AMQCLCHL.TAB
</config-property>
<config-property name="transportType">
CLIENT
</config-property>
<config-property name="queueManager">
*EAIGRP
</config-property>
<security>
<security-domain-and-application>JmsXARealm</security-domain-and-application>
</security>
<recovery>
<recover-credential>
<user-name>mqadmin</user-name>
<password>Mqadmin@123</password>
</recover-credential>
</recovery>
</connection-definition>
</connection-definitions>
<admin-objects>
<admin-object class-name="com.ibm.mq.connector.outbound.MQQueueProxy" jndi-name="jndi/RechareReqQ" pool-name="MQ.QUEUE.NAME">
<config-property name="baseQueueName">
RECHARGE.REQUEST
</config-property>
<config-property name="baseQueueManagerName">
EAI.QM
</config-property>
</admin-object>
</admin-objects>
</resource-adapter>
</resource-adapters>
</subsystem>
|
Thanks ... |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
