Author |
Message
|
hhoang |
Posted: Mon Aug 16, 2004 6:18 am Post subject: The Connection Pool Manager could not allocate a Managed Con |
|
|
Novice
Joined: 14 Nov 2002 Posts: 21
|
We get this message after the network went down.
When then network come back up, we still receive alot of these messages. It seem like the WebSphere Application Server server failed to reconnect to MQ. As result, WebSphere failed to send the messages.
Here is the Connection Pool setting for the WebSphere MQ Queue Connection Factories:
Connection Timeout: 180
Max Connections: 500
Min Connections: 1
Reap Time: 180
Unused Timeout: 300
Aged Timeout: 600
Purge Policy: EntirePool
Do you have any ideas?
Thanks,
Hai
RecordId: Rec_428
ComponentId: Application Server
ProcessId: 2236
ThreadId: 1f6729a8
SourceId: com.ibm.ejs.j2c.ConnectionManager
ClassName:
MethodName:
Manufacturer: IBM
Product: WebSphere
Version: Platform 5.1 [BASE 5.1.0 b0344.02]
ServerName: ccswas02\ccswas02\server1
TimeStamp: 8/16/2004 9:07:32.718000000
UnitOfWork:
Severity: 1
Category: ERROR
PrimaryMessage: J2CA0020E: The Connection Pool Manager could not allocate a Managed Connection: java.lang.IllegalStateException: Internal Error: cannot find the PoolManager Reference.
at com.ibm.ejs.j2c.ConnectionManager.allocateMCWrapper(ConnectionManager.java(Compiled Code))
at com.ibm.ejs.j2c.ConnectionManager.associateConnection(ConnectionManager.java(Compiled Code))
at com.ibm.ejs.jms.JMSSessionHandle.getOpenSession(JMSSessionHandle.java(Compiled Code))
at com.ibm.ejs.jms.JMSSessionHandle.createTextMessage(JMSSessionHandle.java(Compiled Code))
at com.coair.java.logging.JMSAppender.append(JMSAppender.java(Compiled Code))
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java(Compiled Code))
at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java(Compiled Code))
at org.apache.log4j.Category.callAppenders(Category.java(Compiled Code))
at org.apache.log4j.Category.forcedLog(Category.java(Inlined Compiled Code))
at org.apache.log4j.Category.info(Category.java(Compiled Code))
at com.coair.java.logging.SetupServlet.destroy(SetupServlet.java:51)
at com.ibm.ws.webcontainer.servlet.StrictServletInstance.doDestroy(StrictServletInstance.java:159)
at com.ibm.ws.webcontainer.servlet.StrictLifecycleServlet._destroy(StrictLifecycleServlet.java:218)
at com.ibm.ws.webcontainer.servlet.IdleServletState.destroy(StrictLifecycleServlet.java:316)
at com.ibm.ws.webcontainer.servlet.StrictLifecycleServlet.destroy(StrictLifecycleServlet.java:119)
at com.ibm.ws.webcontainer.servlet.ServletInstance.destroy(ServletInstance.java:470)
at com.ibm.ws.webcontainer.servlet.ServletManager.removeServlet(ServletManager.java:153)
at com.ibm.ws.webcontainer.webapp.WebAppServletManager.removeServlet(WebAppServletManager.java:410)
at com.ibm.ws.webcontainer.servlet.ServletManager.removeAllServlets(ServletManager.java:186)
at com.ibm.ws.webcontainer.webapp.WebApp.destroy(WebApp.java:380)
at com.ibm.ws.webcontainer.srt.WebGroup.shutdown(WebGroup.java:1334)
at com.ibm.ws.webcontainer.WebContainer.removeWebApplication(WebContainer.java:1030)
at com.ibm.ws.runtime.component.WebContainerImpl.uninstall(WebContainerImpl.java:147)
at com.ibm.ws.runtime.component.WebContainerImpl.stop(WebContainerImpl.java:367)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.stop(ApplicationMgrImpl.java:432)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStop(DeployedApplicationImpl.java:792)
at com.ibm.ws.runtime.component.DeployedModuleImpl.stop(DeployedModuleImpl.java:381)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.stop(DeployedApplicationImpl.java:612)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.stopApplication(ApplicationMgrImpl.java:329)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.stop(ApplicationMgrImpl.java:319)
at com.ibm.ws.runtime.component.ContainerImpl.stopComponents(ContainerImpl.java:577)
at com.ibm.ws.runtime.component.ContainerImpl.stop(ContainerImpl.java:417)
at com.ibm.ws.runtime.component.ApplicationServerImpl.stop(ApplicationServerImpl.java:149)
at com.ibm.ws.runtime.component.ContainerImpl.stopComponents(ContainerImpl.java:577)
at com.ibm.ws.runtime.component.ContainerImpl.stop(ContainerImpl.java:417)
at com.ibm.ws.runtime.component.ServerImpl.stop(ServerImpl.java:205)
at com.ibm.ws.runtime.component.ServerCollaborator$1.run(ServerCollaborator.java:413)
at com.ibm.ws.security.auth.ContextManagerImpl.contact admin(ContextManagerImpl.java:1705)
at com.ibm.ws.security.auth.ContextManagerImpl.contact admin(ContextManagerImpl.java:1582)
at com.ibm.ws.runtime.component.ServerCollaborator$ShutdownHook.run(ServerCollaborator.java:396)
at com.ibm.ws.runtime.component.ServerCollaborator$StopAction.alarm(ServerCollaborator.java:373)
at com.ibm.ejs.util.am._Alarm.run(_Alarm.java(Compiled Code))
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java(Compiled Code))
.
ExtendedMessage:
RawData: |
|
Back to top |
|
 |
jefflowrey |
Posted: Mon Aug 16, 2004 7:02 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002 Posts: 19981
|
What happens when you restart WAS? _________________ I am *not* the model of the modern major general. |
|
Back to top |
|
 |
hhoang |
Posted: Mon Aug 16, 2004 7:41 am Post subject: Everything works fine if restart WAS |
|
|
Novice
Joined: 14 Nov 2002 Posts: 21
|
We've a bunch of WAS running on a farm. I don't want to go and restart every single WAS when the network or MQ went down. I thought when you set the Aged Timeout and Purge entirepool in the connection pool should correct the problem but I was wrong. |
|
Back to top |
|
 |
fjb_saper |
Posted: Mon Aug 16, 2004 9:42 am Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
Quote: |
We've a bunch of WAS running on a farm. I don't want to go and restart every single WAS when the network or MQ went down. I thought when you set the Aged Timeout and Purge entirepool in the connection pool should correct the problem but I was wrong. |
It really depends what you want to do with your applications. But I would deem that scenario dangerous. You would want to forward / deploy your JMS beans only on one or two servers. These should have their own Qmgrs (==> mqserver) on the same box. This way you do not have to restart the WAS for a connectivity problem.
Remember 2 things:
WAS uses MDBs for connection to the qmgrs and the queues on the receiving side. This means that if the connection goes down you WILL have to restart WAS. The alternative would be to set an exception listener to the queues serviced by the MDB's and destroy and reinstaciate the MDB's on exception. Blast there goes another connection / per queue!
There is no graceful way to put it. The usual design is that if this is that important to you, you better put your WAS on top of MQ server.
Enjoy |
|
Back to top |
|
 |
hhoang |
Posted: Thu Aug 19, 2004 12:54 pm Post subject: Update: Lower max connections from 500 to 100 |
|
|
Novice
Joined: 14 Nov 2002 Posts: 21
|
If I lowered the max connections from 500 to 100, the error message goes away. I guess this problem maybe because MQ can not accept more connections then what is available. Is there away for me to check to see how many connections MQ makes available? |
|
Back to top |
|
 |
fjb_saper |
Posted: Thu Aug 19, 2004 2:47 pm Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
Try
The default is 256.
Remember that after changing this you will have to restart the qmgr for the change to take.
Enjoy |
|
Back to top |
|
 |
hhoang |
Posted: Fri Aug 20, 2004 7:22 am Post subject: |
|
|
Novice
Joined: 14 Nov 2002 Posts: 21
|
I thought lowered the max connection pool help solve the problem but it was not the case. I did a test yesterday by restart the MQ machine and I start seeing the same error messages on WAS site.
Basically, WAS holds a live connections to MQ. If MQ is down or the network is down, the connection states become invalid. I guess, we can start using MQ API directly and without using connection pool can solve this problem. What do you think of the drawbacks beside performance? |
|
Back to top |
|
 |
fjb_saper |
Posted: Fri Aug 20, 2004 9:37 am Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
Connection pool or no connection pool you will be hitting the same problems: network connectivity.
One way around it would be to put an exception listener on the queues that have an MDB listening to them.
Now the only way to bring real stability to your implementation would be to put your was on the same machine as an MQ Server and use a local Queue Manager. This way the network interruptions are automatically handled by the channels.
This is a case were more bucks to pay means less worries...'
Now you are talking about a bunch of WAS ...
Maybe you are suffering from net collisions in an environment were traffic is exeptionally high.
Have you tried setting up multible MQServers and linking a couple of WAS to the MQSever on a special network (reserved for this kind of traffic?)
Enjoy |
|
Back to top |
|
 |
|