Author |
Message
|
nukalas2010 |
Posted: Sun Apr 28, 2013 5:59 am Post subject: EG getting restarting due to operating system stack overflow |
|
|
 Master
Joined: 04 Oct 2010 Posts: 220 Location: Somewhere in the World....
|
Dears,
We are facing one terrific issue, that our execution group is getting restarted with error called " java/lang/StackOverflowError: operating system stack overflow".
We are using the Message Broker Verison 7.0.0.4 on AIX 64 - bit.
Here in the code, depend on the request paramters we have to another service repeatedly in a loop.
Here is the exception we found in stderr logs.
Code: |
Exception in thread "Thread-27" java/lang/StackOverflowError: operating system stack overflow
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469)
at com/ibm/broker/plugin/MbOutputTerminal._propagate (Native Method)
at com/ibm/broker/plugin/MbOutputTerminal.propagate (MbOutputTerminal.java:107)
at com/ibm/broker/transportheadernodes/mq/MQHeaderNode.evaluate (MQHeaderNode.java:1337)
at com/ibm/broker/plugin/MbNode.evaluate (MbNode.java:1469) |
we referred to another related post here
http://www.mqseries.net/phpBB/viewtopic.php?t=48422
We tried the same mentioned over there related to IBM_JAVA_OPTIONS, but no use. Could anyone please throw some ligth on this.
Thanks, |
|
Back to top |
|
 |
fjb_saper |
Posted: Sun Apr 28, 2013 8:59 am Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
It's the loop.
Loops are at your own risk. Don't do them. Use a code loop with propagate inside...and a way of terminating the loop (usually in the Environment).
Have fun  _________________ MQ & Broker admin |
|
Back to top |
|
 |
nukalas2010 |
Posted: Sun Apr 28, 2013 12:29 pm Post subject: |
|
|
 Master
Joined: 04 Oct 2010 Posts: 220 Location: Somewhere in the World....
|
Yeah... but the same bar file is working fine without any issues on another server... all the configurations are same... If it's a code or loop issue.. it should fail there too .... |
|
Back to top |
|
 |
ramires |
Posted: Sun Apr 28, 2013 2:01 pm Post subject: |
|
|
Knight
Joined: 24 Jun 2001 Posts: 523 Location: Portugal - Lisboa
|
Recheck is the severs are configured equal, look for Out of Memory exceptions messages.
Regards |
|
Back to top |
|
 |
mqjeff |
Posted: Mon Apr 29, 2013 3:49 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008 Posts: 17447
|
nukalas2010 wrote: |
Yeah... but the same bar file is working fine without any issues on another server... all the configurations are same... If it's a code or loop issue.. it should fail there too .... |
Only if you send the same exact message. |
|
Back to top |
|
 |
Esa |
Posted: Mon Apr 29, 2013 6:02 am Post subject: |
|
|
 Grand Master
Joined: 22 May 2008 Posts: 1387 Location: Finland
|
nukalas2010 wrote: |
Yeah... but the same bar file is working fine without any issues on another server... all the configurations are same... If it's a code or loop issue.. it should fail there too .... |
1. Remove all other flows from the execution group
2. Remove all other execution groups from the broker.
3. Then send exactly the same message.
If you are lucky, this will help you to loop a couple of milliseconds more
MQHeader node memory leak was corrected in some fix pack.
I cannot check which one because IBM seems to be in process of replacing Message Broker product pages with IBM Integration Bus product pages.
The first part of the project has already been successfully executed: remove Message Broker product pages.
However, your problem is not caused by the memory leak but the very wrong way of looping. |
|
Back to top |
|
 |
nukalas2010 |
Posted: Sat May 04, 2013 10:41 pm Post subject: |
|
|
 Master
Joined: 04 Oct 2010 Posts: 220 Location: Somewhere in the World....
|
Thanks Esa - for your valuable suggestions..
any how the issue was solved after increasing the jvmJavaOSStackSize on that EG to 3Megs. Because, on AIX the default OS stack size should be 2mb.
This is not a code issue...
Thanks to IBM  |
|
Back to top |
|
 |
Vitor |
Posted: Sun May 05, 2013 6:05 am Post subject: |
|
|
 Grand High Poobah
Joined: 11 Nov 2005 Posts: 26093 Location: Texas, USA
|
nukalas2010 wrote: |
This is not a code issue... |
You need to increase the stack to 3Mb just to get the code to not blow up?
What will you do when you get a message which increases the stack requirement? Increase it again?
Some might call that a code problem. _________________ Honesty is the best policy.
Insanity is the best defence. |
|
Back to top |
|
 |
kimbert |
Posted: Mon May 06, 2013 10:52 am Post subject: |
|
|
 Jedi Council
Joined: 29 Jul 2003 Posts: 5542 Location: Southampton
|
I agree with Vitor. It is not a problem in IBM code, but there is a problem in the design of your message flow.
I'm glad that you got it working for now, but you should at least be aware that it might break again in the future. |
|
Back to top |
|
 |
nukalas2010 |
Posted: Tue May 07, 2013 2:24 am Post subject: |
|
|
 Master
Joined: 04 Oct 2010 Posts: 220 Location: Somewhere in the World....
|
kimbert wrote: |
I agree with Vitor. It is not a problem in IBM code, but there is a problem in the design of your message flow.
I'm glad that you got it working for now, but you should at least be aware that it might break again in the future. |
ok sure... thankyou kimbert, vitor... |
|
Back to top |
|
 |
|