ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Failed to allocate memory - Abend File

Post new topic  Reply to topic
 Failed to allocate memory - Abend File « View previous topic :: View next topic » 
Author Message
CuriCAT
PostPosted: Thu Nov 06, 2008 5:51 pm    Post subject: Failed to allocate memory - Abend File Reply with quote

Voyager

Joined: 26 Sep 2006
Posts: 82
Hi, I need a help to find out the reason for memory allocation error.

My flow used to run fine, but abruptly EG gets restarted. And I can see Abend file produced. but I am not able to understand what exactly caused the problem by looking at abend file, can you please help me to understand this, thanks.

I can only see Fail to allocate memory, it doesn't say where/which node throws this error.


Operating System :- AIX |
| Version :- 5 |
| Release :- 3

Message Broker 6007

abend record for pid 1609764 tid 8482 time Thu Nov 6 06:32:54 2008
File: /build/S600_P/src/CommonServices/Unix/ImbAbend.cpp
Line: 1958
Function: ImbAbend::newHandler
---- Inserts ----
Failed to allocate memory
-----------------
----------------------------- Stack dump for current thread ( 8482)
(malloc allowed)
(0xd79f39d8+0x00000204) dumpCurrentStacksToFile__8ImbAbendFPCvi [/opt/IBM/mqsi/6.0/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd79f15e4+0x00001198) dumpToFile__8ImbAbendFPcPCciPCcRCQ2_8ImbAbend6InsertRCQ2_8ImbAbend6InsertRCQ2_8ImbAbend6InsertRCQ2_8ImbAbend6InsertRCQ2_8ImbAbend6InsertRCQ2_8ImbAbend6InsertPCv [/opt/IBM/mqsi/6.0/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd795cfc0+0x00000180) logSystemErrorInternal__8ImbAbendFPCciPCcRCQ2_8ImbAbend6InsertRCQ2_8ImbAbend6InsertRCQ2_8ImbAbend6InsertRCQ2_8ImbAbend6InsertRCQ2_8ImbAbend6InsertRCQ2_8ImbAbend6InsertPCvb [/opt/IBM/mqsi/6.0/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd793c314+0x00000110) logSystemErrorAndTerminateProcessInternal__8ImbAbendFPCciPCcRCQ2_8ImbAbend6InsertRCQ2_8ImbAbend6InsertRCQ2_8ImbAbend6InsertRCQ2_8ImbAbend6InsertRCQ2_8ImbAbend6InsertRCQ2_8ImbAbend6InsertPCvb [/opt/IBM/mqsi/6.0/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd7ac3948+0x0000006c) newHandler__8ImbAbendFv [/opt/IBM/mqsi/6.0/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd07ddf00+0x0000010c) __nw__FUl [/usr/vacpp/lib/libC.a(ansicore_32.o)]
(0xd07ddeb8+0x00000010) __vn__FUl [/usr/vacpp/lib/libC.a(ansicore_32.o)]
(0xd7c10828+0x00000184) appendInternal__10ImbWstringFPCUsUii_23_18 [/opt/IBM/mqsi/6.0/lib/libPersonality.a(libPersonality.a.so)]
(0xd7cfa894+0x0000002c) append__10ImbWstringFPCUsUi [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7cfa800+0x00000044) __apl__10ImbWstringFRC10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7cfc884+0x000001ec) logTreeAsStringInner__16ImbSyntaxElementFR10ImbWstring10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7cfc884+0x00000288) logTreeAsStringInner__16ImbSyntaxElementFR10ImbWstring10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7cfc884+0x00000288) logTreeAsStringInner__16ImbSyntaxElementFR10ImbWstring10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7cfc884+0x00000288) logTreeAsStringInner__16ImbSyntaxElementFR10ImbWstring10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7cfc884+0x00000288) logTreeAsStringInner__16ImbSyntaxElementFR10ImbWstring10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7cfc884+0x00000288) logTreeAsStringInner__16ImbSyntaxElementFR10ImbWstring10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7cfc884+0x00000288) logTreeAsStringInner__16ImbSyntaxElementFR10ImbWstring10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7cfc884+0x00000288) logTreeAsStringInner__16ImbSyntaxElementFR10ImbWstring10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7cfc884+0x00000288) logTreeAsStringInner__16ImbSyntaxElementFR10ImbWstring10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7cfc884+0x00000288) logTreeAsStringInner__16ImbSyntaxElementFR10ImbWstring10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7cfc884+0x00000288) logTreeAsStringInner__16ImbSyntaxElementFR10ImbWstring10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7cfc884+0x00000288) logTreeAsStringInner__16ImbSyntaxElementFR10ImbWstring10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7cfa95c+0x000000d4) logTreeAsString__16ImbSyntaxElementFR10ImbWstring10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xda0df19c+0x0000008c) logTreeAsString__Q2_10ImbMessage10ReadCursorFR10ImbWstring [/opt/IBM/mqsi/6.0/lib/libImbRdl.a(libImbRdl.a.so)]
(0xda0df020+0x000000c8) logToString2__19SqlCursorEvalResultFR10ImbWstring [/opt/IBM/mqsi/6.0/lib/libImbRdl.a(libImbRdl.a.so)]
(0xda54709c+0x000001b4) logToString__13SqlEvalResultFR10ImbWstring [/opt/IBM/mqsi/6.0/lib/libImbRdl.a(libImbRdl.a.so)]
(0xda558ca0+0x00000460) evaluate__21SqlTraceNodeInterfaceFRC18ImbMessageAssemblyR10ImbWstring [/opt/IBM/mqsi/6.0/lib/libImbRdl.a(libImbRdl.a.so)]
(0xdb873520+0x000000dc) evaluate__12ImbTraceNodeFRC18ImbMessageAssemblyPC19ImbDataFlowTerminal [/opt/IBM/mqsi/6.0/lil/imbdfsql.lil]
(0xd7dd5644+0x000001ec) evaluate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7dd4ea0+0x000000cc) propagateInner__19ImbDataFlowTerminalFRC18ImbMessageAssemblyP19ImbDataFlowTerminal [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd8030b7c+0x000004a4) propagate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xdb8b9030+0x00000558) propagateToTerminal__14ImbComputeNodeFRC8ImbValueiRC18ImbMessageAssemblyRC18ImbMessageAssembly [/opt/IBM/mqsi/6.0/lil/imbdfsql.lil]
(0xda53d454+0x0000006c) propagateToTerminal__19SqlComputeInterfaceCFRC8ImbValueiRC18ImbMessageAssemblyR18ImbMessageAssembly [/opt/IBM/mqsi/6.0/lib/libImbRdl.a(libImbRdl.a.so)]
(0xda4b4260+0x00000db0) execute__21SqlPropagateStatementCFR18SqlStatementResult [/opt/IBM/mqsi/6.0/lib/libImbRdl.a(libImbRdl.a.so)]
(0xda3f629c+0x000000d8) execute__17SqlStatementGroupCFR18SqlStatementResult [/opt/IBM/mqsi/6.0/lib/libImbRdl.a(libImbRdl.a.so)]
(0xda506c0c+0x00000960) execute__14SqlIfStatementCFR18SqlStatementResult [/opt/IBM/mqsi/6.0/lib/libImbRdl.a(libImbRdl.a.so)]
(0xda3f629c+0x000000d8) execute__17SqlStatementGroupCFR18SqlStatementResult [/opt/IBM/mqsi/6.0/lib/libImbRdl.a(libImbRdl.a.so)]
(0xda4880d8+0x00000210) execute__15SqlCompoundStmtCFR18SqlStatementResult [/opt/IBM/mqsi/6.0/lib/libImbRdl.a(libImbRdl.a.so)]
(0xda312380+0x000001c8) execute__10SqlRoutineCFR18SqlStatementResult [/opt/IBM/mqsi/6.0/lib/libImbRdl.a(libImbRdl.a.so)]
(0xda4cc698+0x00000144) execute__9SqlModuleCFR18SqlStatementResult [/opt/IBM/mqsi/6.0/lib/libImbRdl.a(libImbRdl.a.so)]
(0xda4c4388+0x00000024) execute__9SqlSchemaCFR18SqlStatementResult_148_23 [/opt/IBM/mqsi/6.0/lib/libImbRdl.a(libImbRdl.a.so)]
(0xda53d690+0x0000054c) evaluate__19SqlComputeInterfaceFRC18ImbMessageAssemblyR18ImbMessageAssembly [/opt/IBM/mqsi/6.0/lib/libImbRdl.a(libImbRdl.a.so)]
(0xdb88275c+0x00000398) evaluate__14ImbComputeNodeFRC18ImbMessageAssemblyPC19ImbDataFlowTerminal [/opt/IBM/mqsi/6.0/lil/imbdfsql.lil]
(0xd7dd5644+0x000001ec) evaluate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7dd4ea0+0x000000cc) propagateInner__19ImbDataFlowTerminalFRC18ImbMessageAssemblyP19ImbDataFlowTerminal [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd8030b7c+0x000004a4) propagate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xdb873520+0x0000034c) evaluate__12ImbTraceNodeFRC18ImbMessageAssemblyPC19ImbDataFlowTerminal [/opt/IBM/mqsi/6.0/lil/imbdfsql.lil]
(0xd7dd5644+0x000001ec) evaluate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7dd4ea0+0x000000cc) propagateInner__19ImbDataFlowTerminalFRC18ImbMessageAssemblyP19ImbDataFlowTerminal [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd8030b7c+0x000004a4) propagate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xda819000+0x000000ec) evaluate__12ImbLabelNodeFRC18ImbMessageAssemblyPC19ImbDataFlowTerminal [/opt/IBM/mqsi/6.0/lil/imbdfbas.lil]
(0xda7dfb34+0x00000dc8) evaluate__13ImbRouterNodeFRC18ImbMessageAssemblyPC19ImbDataFlowTerminal [/opt/IBM/mqsi/6.0/lil/imbdfbas.lil]
(0xd7dd5644+0x000001ec) evaluate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7dd4ea0+0x000000cc) propagateInner__19ImbDataFlowTerminalFRC18ImbMessageAssemblyP19ImbDataFlowTerminal [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd8030b7c+0x000004a4) propagate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xdb87cc40+0x000009f4) evaluate__13ImbFilterNodeFRC18ImbMessageAssemblyPC19ImbDataFlowTerminal [/opt/IBM/mqsi/6.0/lil/imbdfsql.lil]
(0xd7dd5644+0x000001ec) evaluate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7dd4ea0+0x000000cc) propagateInner__19ImbDataFlowTerminalFRC18ImbMessageAssemblyP19ImbDataFlowTerminal [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd8030b7c+0x000004a4) propagate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xdb87cc40+0x00000a90) evaluate__13ImbFilterNodeFRC18ImbMessageAssemblyPC19ImbDataFlowTerminal [/opt/IBM/mqsi/6.0/lil/imbdfsql.lil]
(0xd7dd5644+0x000001ec) evaluate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd7dd4ea0+0x000000cc) propagateInner__19ImbDataFlowTerminalFRC18ImbMessageAssemblyP19ImbDataFlowTerminal [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd8030b7c+0x000004a4) propagate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xdb362ff8+0x00000128) Java_com_ibm_broker_plugin_MbOutputTerminal__1propagate [/opt/IBM/mqsi/6.0/lib/libimbjplg.a]
(0x5e4da3dc+??????] <no name available> [/opt/IBM/mqsi/6.0/lib/libimbjplg.a]
----------------------------------------------------------------------
Back to top
View user's profile Send private message Send e-mail Yahoo Messenger
Gaya3
PostPosted: Thu Nov 06, 2008 7:25 pm    Post subject: Reply with quote

Jedi

Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
it could be an OS issue

I suggest you to raise a PMR at the earliest.
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die
Back to top
View user's profile Send private message
sridhsri
PostPosted: Sat Nov 08, 2008 5:03 pm    Post subject: Reply with quote

Master

Joined: 19 Jun 2008
Posts: 297
I would recommend that you do a little debugging yourself before you raise the PMR. I don't necessarily agree that any "failed to allocate memory" abend is something to do with the OS.

You should start be inspecting the flow you believe is causing the problem. Are there are custom nodes, JCN nodes you have ? Usually these would be the first place I would look to see if the objects are being cleared.
Back to top
View user's profile Send private message
CuriCAT
PostPosted: Sat Nov 08, 2008 9:01 pm    Post subject: Reply with quote

Voyager

Joined: 26 Sep 2006
Posts: 82
The flow consists of MQ nodes, HTTP nodes, XSL Transformation node, Java compute node, trance nodes. I am not processing any huge files. Maximum size of the message is about 30 kb. if I try to test it number of times Flow runs okay. so not able to trace exactly whats the problem. but It blows randomly 4 or 5 times in a day. the abend file doesn't say exactly which node blows.

Quote:

0xd7dd4ea0+0x000000cc) propagateInner__19ImbDataFlowTerminalFRC18ImbMessageAssemblyP19ImbDataFlowTerminal [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xd8030b7c+0x000004a4) propagate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)]
(0xdb362ff8+0x00000128) Java_com_ibm_broker_plugin_MbOutputTerminal__1propagate [/opt/IBM/mqsi/6.0/lib/libimbjplg.a]
Back to top
View user's profile Send private message Send e-mail Yahoo Messenger
smdavies99
PostPosted: Sun Nov 09, 2008 12:37 am    Post subject: Sep 1 - Raise a PMR Reply with quote

Jedi Council

Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
Step 2 put user trace on the EG with a HUGE size ( -c 65536 or larger ) and let it rip. When it abends, you will have some idea about what the EG was doing at the time of the abend.
Step 3 save all the messages that get sent through and try to develop a pattern of what is causing the abend.

Also
If you have not done this already, try putting each flow in the EG in its own EG. This may isolate it down to one flow

Then if directed by IBM support take a service trace of the EG and get it to abend.

Remember the old mantra of Support Engineers,

Making the problem repeatable is often more than 50% of the way to solving the problem
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995

Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions.
Back to top
View user's profile Send private message
JosephGramig
PostPosted: Sun Nov 09, 2008 7:03 am    Post subject: Reply with quote

Grand Master

Joined: 09 Feb 2006
Posts: 1244
Location: Gold Coast of Florida, USA
I would immediately suspect the JavaCompute node and the code there in.

Before you open a PMR, make sure that MQ is at the latest maintenance, Message Brokers is too, the OS is (AIX in this case), and xlC++ is too (9.0.0.3 runtime I think).

To view xlC level, try 'lslpp -l|grep xlC'.

Always run with EXTSHM=ON in the environment of mqm and the service ID of any application that uses WebSphere MQ (doesn't hurt and might help).
Back to top
View user's profile Send private message AIM Address
joebuckeye
PostPosted: Mon Nov 10, 2008 7:49 am    Post subject: Reply with quote

Partisan

Joined: 24 Aug 2007
Posts: 365
Location: Columbus, OH
In the JCN make sure you call clearMessage() on any message you worked on in the code (after you propogate it of course).
Back to top
View user's profile Send private message
CuriCAT
PostPosted: Tue Nov 11, 2008 1:12 pm    Post subject: Reply with quote

Voyager

Joined: 26 Sep 2006
Posts: 82
raised a PMR, I have received a soution from IBM - I have to increace maximum memory alocation by using following set of commands.

mqsistop brokerName
export LDR_CNTRL=MAXDATA=0x60000000
mqsistart brokerName


My concern is that, this will increase maximum memory size for all the EGs in the broker, I want to do this only for a specific EG.
I searched in information center and Message Broker Administoration. I wonder we have such an option.

I want todo this in AIX system. Thanks.
Back to top
View user's profile Send private message Send e-mail Yahoo Messenger
sridhsri
PostPosted: Wed Nov 12, 2008 6:01 am    Post subject: Reply with quote

Master

Joined: 19 Jun 2008
Posts: 297
I am not sure if it is possible for you to do that. I'm sure you already know this but...the mqsichangeproperties command allows you to tweak the JVM heap sizes for each EG (http://publib.boulder.ibm.com/infocenter/wmbhelp/v6r1m0/index.jsp?topic=/com.ibm.etools.mft.doc/au16640_.htm)
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Failed to allocate memory - Abend File
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
  
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.