| Author |
Message
|
| krypton |
 Posted: Tue Apr 11, 2017 6:24 am Post subject: JBOSS to MQ Error - MQJCA1011: Failed to allocate a JMS conn |
 |
|
Disciple
Joined: 14 Mar 2010
Posts: 186
|
we are using JBOSS EAP 6.4 and suddenly after the weekend reboot/Linux server patching, one of the application start getting this error.
We are not sure why this error has occurred as we have not changed anything in configuration.
| Quote: |
2017-04-10 13:45:12,230 ERROR [com.ad.platform.messaging.exception.EAIExceptionUtil] JMS Exception Occured:: com.ibm.msg.client.jms.DetailedJMSException: MQJCA1011: Failed to allocate a JMS connection.
An internal error caused an attempt to allocate a connection to fail.
Caused by: javax.resource.ResourceException: IJ000655: No managed connections available within configured blocking timeout (30000 [ms])
|
has anyone seen this error?
_________________
Dreams are not something which you watch when you are asleep,it is something which doesn't let you sleep. |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Apr 11, 2017 7:26 am Post subject: Re: JBOSS to MQ Error - MQJCA1011: Failed to allocate a JMS |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| krypton wrote: |
| has anyone seen this error? |
Typically when the version of JBoss isn't supported by the Linux OS version.
Are you sure that JBoss level / feature / function is supported by your new Linux level?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Apr 11, 2017 7:27 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
You might also want to check what Linux features (if any) are reset to defaults by applying the patch. You may need to reapply a Linux configuration.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| krypton |
| Posted: Tue Apr 11, 2017 8:03 am Post subject: |
|
|
Disciple
Joined: 14 Mar 2010
Posts: 186
|
| Quote: |
Typically when the version of JBoss isn't supported by the Linux OS version.
|
| Quote: |
You might also want to check what Linux features (if any) are reset to defaults by applying the patch. You may need to reapply a Linux configuration.
|
Jeff, I didn't look at the issue that way because we have the LINUX patching done in PRD servers as well , but the issue is happening only in our UAT server and nothing happening in PRD.[/quote]
_________________
Dreams are not something which you watch when you are asleep,it is something which doesn't let you sleep. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Apr 11, 2017 8:07 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Ouch. That hurts
| krypton wrote: |
| I didn't look at the issue that way because we have the LINUX patching done in PRD servers as well , but the issue is happening only in our UAT server and nothing happening in PRD. |
If your UAT boxes are configured the same way as your PRD boxes, you either have some very large UAT test cases or ridiculously small production volumes.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| krypton |
| Posted: Tue Apr 11, 2017 8:12 am Post subject: |
|
|
Disciple
Joined: 14 Mar 2010
Posts: 186
|
| Quote: |
krypton wrote:
Jeff
Ouch. That hurts
|
Vitor, My apology for that.
| Quote: |
If your UAT boxes are configured the same way as your PRD boxes, you either have some very large UAT test cases or ridiculously small production volumes.
|
I know what you are trying to imply here, but somehow my instinct saying that this issue is either to do with JBOSS or MQ configuration and not related to LINUX configuration.
my intuition basis is I don't see anywhere mention OS resource problem in the error message thrown by JBOSS.
_________________
Dreams are not something which you watch when you are asleep,it is something which doesn't let you sleep. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Apr 11, 2017 8:30 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| krypton wrote: |
| somehow my instinct saying that this issue is either to do with JBOSS or MQ configuration and not related to LINUX configuration. |
Then you simply need to compare the JBoss configuration in PRD with the one in UAT (which you assert to be identical) and highlight the differences.
Unless you've already done that and found them to be the same. Which then leads my instinct back to it working before you patched the server and failing afterwards. Especially as I'm quietly confident you didn't change the JBoss settings as part of the Linux patch.
| krypton wrote: |
| my intuition basis is I don't see anywhere mention OS resource problem in the error message thrown by JBOSS |
At no time have I said this is a resource problem. I said configuration problem.
You also haven't investigated (or at least posted) the linked exception from JMS. What does MQ say is going wrong? For all you know from the information posted, the Linux patch has changed a setting in the OS IP stack and the UAT box can't make a network connection.
And if you tell me PRD & UAT have the same network settings, I'll laugh then make tutting noises at your network security people.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| krypton |
| Posted: Tue Apr 11, 2017 10:26 am Post subject: |
|
|
Disciple
Joined: 14 Mar 2010
Posts: 186
|
| Quote: |
| You also haven't investigated (or at least posted) the linked exception from JMS. |
Vitor,
Below is the linked exception we pull from application
| Quote: |
| You also haven't investigated (or at least posted) the linked exception from JMS. |
Vitor,
Below is the linked exception we pull from application
2017-04-11 09:11:17,730 ERROR [com.cf.platform.messaging.exception.EAIExceptionUtil] JMS Exception Occured:: com.ibm.msg.client.jms.DetailedJMSException: MQJCA1011: Failed to allocate a JMS connection.
An internal error caused an attempt to allocate a connection to fail.
See the linked exception for details of the failure.
at com.ibm.mq.connector.services.JCAExceptionBuilder.buildException(JCAExceptionBuilder.java:146)
at com.ibm.mq.connector.services.JCAExceptionBuilder.buildException(JCAExceptionBuilder.java:105)
at com.ibm.mq.connector.outbound.ConnectionFactoryImpl.createManagedJMSConnection(ConnectionFactoryImpl.java:229)
at com.ibm.mq.connector.outbound.ConnectionFactoryImpl.createConnectionInternal(ConnectionFactoryImpl.java:153)
at com.ibm.mq.connector.outbound.ConnectionFactoryImpl.createConnection(ConnectionFactoryImpl.java:138)
at com.ibm.mq.connector.outbound.ConnectionFactoryImpl.createConnection(ConnectionFactoryImpl.java:123)
at com.cf.platform.messaging.connection.EAIJNDIConnection.createConnection(EAIJNDIConnection.java:94)
at com.cf.platform.messaging.connection.EAIJNDIConnection.retry(EAIJNDIConnection.java:127)
at com.cf.platform.messaging.connection.EAIJNDIConnection.getConnection(EAIJNDIConnection.java:77)
at com.cf.platform.messaging.command.SyncCFCommand.execute(SyncCFCommand.java:426)
at com.cf.platform.messaging.util.FSEAIBridge.process(FSEAIBridge.java:102)
at com.abcd.kind.dislike.util.dislikeMQAdaptor.sendMQMessage(dislikeMQAdaptor.java:61)
at com.abcd.kind.dislike.util.dislikeManipulator.makeMQCall(dislikeManipulator.java:304)
at com.abcd.kind.dislike.util.dislikeManipulator.dislikeInquiryTwoCall(dislikeManipulator.java:373)
at com.abcd.kind.dislike.util.dislikeManipulator.toggleInquiryCall(dislikeManipulator.java:347)
at com.abcd.kind.dislike.util.dislikeManipulator.dislikeInquiryOneCall(dislikeManipulator.java:5
at net.autodata.cfweb.resource.impl.jaxb.PinkProfileResourceImpl.inquirePinkProfile(PinkProfileResourceImpl.java:130)
at net.autodata.cfweb.service.impl.dislikeProfileServiceImpl.inquiredislikeProfile(dislikeProfileServiceImpl.java:33)
at com.abcd.services.util.MakkindmentLandingServiceUtil.getdislikeLandingJson(MakkindmentLandingServiceUtil.java:115)
at com.abcd.services.wrapper.abcdWrapperService.dislikeLanding(abcdWrapperService.java:744)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:219)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:745)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:686)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:925)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:856)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:936)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:827)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:734)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:812)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:8
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at net.autodata.cfweb.web.filter.SmcNewMessagesRadarFilter.doFilter(SmcNewMessagesRadarFilter.java:59)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at net.autodata.cfweb.web.filter.AuthenticationFilter.doFilter(AuthenticationFilter.java:45)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at net.autodata.cfweb.web.filter.MultipleLoginFilter.doFilter(MultipleLoginFilter.java:91)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at com.abcd.services.wrapper.abcdCrawlServlet.doFilter(abcdCrawlServlet.java:200)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at com.abcd.services.filter.CrossOriginRequestFilter.doFilter(CrossOriginRequestFilter.java:73)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:231)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149)
at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:150)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:490)
at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:420)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:926)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.resource.ResourceException: IJ000655: No managed connections available within configured blocking timeout (30000 [ms])
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:460)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:457)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:429)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:344)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:317)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:499)
at com.ibm.mq.connector.outbound.ConnectionFactoryImpl.createManagedJMSConnection(ConnectionFactoryImpl.java:194)
... 66 more
_________________
Dreams are not something which you watch when you are asleep,it is something which doesn't let you sleep. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Apr 11, 2017 11:03 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| krypton wrote: |
Vitor,
Below is the linked exception we pull from application |
Then why does it say:
| Code: |
2017-04-11 09:11:17,730 ERROR [com.cf.platform.messaging.exception.EAIExceptionUtil] JMS Exception Occured:: com.ibm.msg.client.jms.DetailedJMSException: MQJCA1011: Failed to allocate a JMS connection.
An internal error caused an attempt to allocate a connection to fail.
[b]See the linked exception for details of the failure.[/b]
|
Also that doesn't look like a linked exception to me - that looks like the main exception.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| krypton |
| Posted: Tue Apr 11, 2017 11:31 am Post subject: |
|
|
Disciple
Joined: 14 Mar 2010
Posts: 186
|
Hi Vitor,
what about the last few lines? aren't they linked exception?
| Quote: |
Caused by: javax.resource.ResourceException: IJ000655: No managed connections available within configured blocking timeout (30000 [ms])
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:460)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:457)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:429)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:344)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:317)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:499)
at com.ibm.mq.connector.outbound.ConnectionFactoryImpl.createManagedJMSConnection(ConnectionFactoryImpl.java:194) |
_________________
Dreams are not something which you watch when you are asleep,it is something which doesn't let you sleep. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Apr 11, 2017 11:42 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| krypton wrote: |
Hi Vitor,
what about the last few lines? aren't they linked exception?
| Quote: |
Caused by: javax.resource.ResourceException: IJ000655: No managed connections available within configured blocking timeout (30000 [ms])
at |
|
I'll defer to anyone who knows more about Java than I do (which is most people) but that's not the format I expect from a linked exception, nor am I aware that the linked list of exceptions are output unless there's specific code in the catch block to traverse the tree. What you've posted looks like the JMS exception; the first entry in the exception list.
Java experts, chime in any time here.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Apr 11, 2017 11:59 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
|
| Back to top |
|
|
| krypton |
| Posted: Tue Apr 11, 2017 12:39 pm Post subject: |
|
|
Disciple
Joined: 14 Mar 2010
Posts: 186
|
.
| Quote: |
https://developer.jboss.org/thread/243331
|
from the link it says connection pool is exhausted, is there any change we can do at MQ end to handle this error?
_________________
Dreams are not something which you watch when you are asleep,it is something which doesn't let you sleep. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Apr 11, 2017 12:41 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
MQ isn't in charge of the connection pool.
JBOSS and your JMS applications are.
Either a) your connection pool is too small, b) your connection pool is keeping connections open too long, c) you have a connection leak in one or more of your JBOSS jms applications, d) some combination of any of these.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
|
|
