| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| BIP2121E and BIP2106E |
« View previous topic :: View next topic » |
| Author |
Message
|
| kirani |
 Posted: Mon Feb 04, 2002 5:23 pm Post subject: |
 |
|
Jedi Knight
Joined: 05 Sep 2001
Posts: 3779
Location: Torrance, CA, USA
|
Hi,
I am getting these errors during runtime on WIN2K broker (MQSI V2.0.2, CSD1). In my message flow I am retrieving 1010 records from db (MS-SQL Server 2000) and storing it in a XML tree. Record size is around 200 bytes. Then, I am looping in my message flow to extract one record at a time from the Input XML tree and then send it to the output queue (for now).
After I process 146 messages, I get BIP2121E and BIP2106E (see description below) and my original message goes to DLQ. I am not using any plug-in in my message flow.
We have around 2GB RAM on our server and paging size is set to 2GB or more. My data size in tree is not so big (200 bytes * 1000 records = 200,000 bytes = 195.31 KB). I don't know what is going wrong here. I verified the data for 146th record, and there is nothing wrong with it.
Does anyone know what is going wrong here? Am I missing something?
Here is the detailed error message,
*******
2002-02-04 09:23:100000000 888 Error BIP2121E: The thread bootstrap code caught an unhandled exception on thread '888'.
An internal software error has occurred in the message broker. Further messages will indicate the effect of this error on the broker's transactions. The exception was issued on thread '888'. Futher information may be in the exception information associated with this message.
Restart the message broker. If the problem continues to occur, then restart the system. If the problem continues to occur, then contact your IBM support center.
2002-02-04 09:23:100000000 888 FatalException BIP2106E: Unable to allocate memory during operation performed by object 'FU_Transform.LeaseTransform.RemProcessed'.
An execution group performed an operation which needed to allocate memory, but the request failed because insufficient memory was available. The execution group process will be stopped and an attempt made to restart it.
Since this is indicative of a shortage of memory, it may be necessary (after performing initial problem determination) to either increase the amount of memory available or reduce the system's requirement for memory. The former might involve increasing the available RAM or increasing the amount paging space. The latter might involve stopping applications which are not required or which have high memory requirements and reducing the amount of broker processes running on the system.
*******
Thanks in advance!
Regards,
Kiran
|
|
| Back to top |
|
 |
| mpuetz |
| Posted: Tue Feb 05, 2002 9:20 am Post subject: |
|
|
Centurion
Joined: 05 Jul 2001
Posts: 149
Location: IBM/Central WebSphere Services
|
Hi Kiran,
it really depends on how you do it whether your 2 megs of RAM
are sufficient to do that.
Are you performing a loop in a single compute node or are you
doing it in an *external* loop copying the message tree every
time you do a select ?
_________________
Mathias Puetz
IBM/Central WebSphere Services
MQ/WMQI Specialist
[ This Message was edited by: mpuetz on 2002-02-05 09:21 ] |
|
| Back to top |
|
|
| kirani |
| Posted: Tue Feb 05, 2002 10:27 am Post subject: |
|
|
Jedi Knight
Joined: 05 Sep 2001
Posts: 3779
Location: Torrance, CA, USA
|
Hi Mathias,
I have 2GM RAM. I am doing these operations in *external* loop.
My Message flow looks like this,
MQInput->Compute1->Filter1 (true terminal)->FlowOrder1(First terminal)->Compute2->MQOutput
->FlowOrder1(Second terminal)->Compute3->Filter1 (input terminal).
ESQL in these nodes is as follows,
Compute1 : Reads records from database table and stores them in a tree. (pass Message & Destination in Advanced Tab)
--------
-- Retrieve records from tbl_data
SET OutputRoot.XML.FU.RecordSet[] = (
SELECT
tbl.f1, tbl.f2, tbl.f3, tbl.f4, ... tbl.fn
from Database.tbl_data as tbl
where
tbl.f1 = InputRoot.XML.FU.TransmitDate AND
tbl.f2 = InputRoot.XML.FU.SrcLocation AND
tbl.f3 = 'P'
);
-- Set count variable
SET OutputDestinationList.RecCount = CARDINALITY(OutputRoot.XML.FU.RecordSet[]);
Filter1 - Checks for more records to process in RecordSet[]
-------
DestinationList.RecCount > 0
Compute2 - Extract first record in the RecordSet[] and send it to output queue. (Copy Data header only)
--------
SET OutputRoot.XML.FU.Data = InputRoot.XML.FU.RecordSet[1];
--SET OutputRoot.XML.FU.RecordSet = NULL;
Compute3 - Copy complete message (pass Message and Destination in Advanced Tab)
--------
-- Remove processed record from the tree
SET OutputRoot.XML.FU.RecordSet[1] = NULL;
-- Decrement count by 1
SET OutputDestinationList.RecCount = InputDestinationList.RecCount - 1;
I am very sure one of these nodes is not releasing the memory.
Thanks,
Kiran
|
|
| Back to top |
|
|
| kirani |
| Posted: Wed Feb 06, 2002 8:57 am Post subject: |
|
|
Jedi Knight
Joined: 05 Sep 2001
Posts: 3779
Location: Torrance, CA, USA
|
For one unit of work (UOW), Broker keeps all copies of the parsed tree in the memory. After the UOW is over, these trees are deleted from the memory. This is the reason I am running out of memory. I have 100's of parsed tree in the memory for one UOW and each tree size is in MegaBytes.
In Summary, if you are trying to build multiple output messages from single input message using some kind of "external" loop, eventually you are going to have memory problem. This will happen if the parsed tree is very large and the number of output messages are high.
You have to be very careful while designing these kind of Message flows.
Regards,
Kiran
|
|
| Back to top |
|
|
| mpuetz |
| Posted: Thu Feb 07, 2002 10:03 am Post subject: |
|
|
Centurion
Joined: 05 Jul 2001
Posts: 149
Location: IBM/Central WebSphere Services
|
Hi,
one more info on this.
Typically when using external loops with a
large repeat count one runs into a stack problem
first, since essentially you are creating a recursive
loop in terms of the underlying MQSI programming.
I.e. when MQSI propagates a message from a output
terminal it *calls* the evaluate function of the
following node. This call returns only when the
following flow-branch ends. Hence the stack of the calling
node is not cleaned up before the subflow which it propagates
to is completed. Thats also why all copies you generate
on the fly won't be cleaned up before that particular branch
has ended.
With 2.0.2 CSD 02 on UNIX you can increase stack size from
its 1MB default value by setting an environment variable
to mitigate this.
With MQSI 2.0.x on Win NT/2K you can use the editbin command
(shipped e.g. with MS Visual C++) to increase the stack
of the dataflowengine.exe
The maximum *heap* memory you can use within a dataflowengine
is platform dependent. On NT and HP-UX you can use up to the
physical amount of memory (+ virtual mem) automatically. On AIX
only upto 512MB. You need a patched dataflowengine from IBM
if you need more.
_________________
Mathias Puetz
IBM/Central WebSphere Services
WebSphere Business Integration Specialist |
|
| Back to top |
|
|
| mpuetz |
| Posted: Thu Feb 07, 2002 10:10 am Post subject: |
|
|
Centurion
Joined: 05 Jul 2001
Posts: 149
Location: IBM/Central WebSphere Services
|
One more trick to saving memory in external loops:
1.) keep the number of node within the loop small.
2.) *externalize* the loop body, if possible.
both are possible if the iteration n does not depend
on the result of n-1. then you can use a flow order node to solve this.
filter(i < maxrec) --(yes)--> floworder --(first)--> do your thing for i ---> output or whatever
floworder --(second)-->compute(i=i+1)------>filter(i>maxrec)--(no)-->end processing
In the i=i+1 node only modify the destinationlist and leave the message unchanged.
Now your the temporary message buffers you need for each transformation
are freed in each loop iteration.
Hope this helps.
_________________
Mathias Puetz
IBM/Central WebSphere Services
MQ/WMQI Specialist
[ This Message was edited by: mpuetz on 2002-02-07 10:12 ]
[ This Message was edited by: mpuetz on 2002-02-07 10:15 ] |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
