| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| IIB v10 dataflow engine taking more than 6GB memory. |
« View previous topic :: View next topic » |
| Author |
Message
|
| ramesh4800 |
 Posted: Fri Mar 30, 2018 1:07 am Post subject: IIB v10 dataflow engine taking more than 6GB memory. |
 |
|
Newbie
Joined: 05 Jan 2015
Posts: 5
|
We are processing large xml files by using soap request nodes.and EG level heap size is 1GB. There is no any additional instances,but still it is taking 6GB memory for dataflow engine.
Why this much 6GB memory is allocated?? it is not releasing although is it completed the process.
my question is why 6GB if we allocated only 1 GB on EG heap.What all the resources using memory???
thanks in advance. |
|
| Back to top |
|
 |
| joebuckeye |
| Posted: Fri Mar 30, 2018 3:46 am Post subject: |
|
|
Partisan
Joined: 24 Aug 2007
Posts: 365
Location: Columbus, OH
|
My very unofficial answer that could be based on old information (since it's been awhile since we had to look at this) is that parts of IIB are written in Java and parts are in C/C++. I believe the EG heap size parameter is for the Java bits.
And once the DataFlowEngine allocates memory it never releases it back to the system because it assumes (not without reason) that if you needed that much memory once that you will need it again so it keeps it.
Again, we haven't had memory issues since v7/v8 so not sure if this is still true in v10 (we skipped v9). |
|
| Back to top |
|
|
| mpong |
| Posted: Fri Mar 30, 2018 2:06 pm Post subject: |
|
|
Disciple
Joined: 22 Jan 2010
Posts: 164
|
How does WebSphere Message Broker (WMB) handle the memory usage within a DataFlowEngine (DFE) Process?
When considering memory usage within a DataFlowEngine process there are two sources that the storage is allocated from, and these are :
1. The DataFlowEngine main memory heap
2. The operating system
When message flow processing requires some storage, then an attempt is first made to allocate the memory block required from the DataFlowEngine's heap. If there is not a large enough contiguous block of storage on the heap, then a request will be made to the operating system to allocate more storage to the DataFlowEngine for the message flow. Once this is done, then this would lead to the DataFlowEngine's heap growing with the additional storage, and the message flow will use this extra storage.
When the message flow has completed its processing, then it issues a "free" on all its storage and these blocks will be returned to the DataFlowEngine's heap ready for allocation to any other message flows of this DataFlowEngine. The storage is never released back to the operating system, because there is actually no programmatic mechanism to perform such an operation. The operating system will not retrieve storage from a process until the process is terminated. Therefore the user will never see the size of the DataFlowEngine process decrease, after it has increased.
When the next message flow runs, then it will make requests for storage, and these will then be allocated from the DataFlowEngine heap as before. Therefore there will be a re-use within the DataFlowEngine's internal storage where possible, minimizing the number of times that the operating system needs to allocate additional storage to the DataFlowEngine process. This would mean there may be some growth observed on DataFlowEngine's memory usage which is of the size of the subsequent allocations for message flow processing. Eventually we would expect the storage usage to plateau, and this situation would occur when the DataFlowEngine has a large enough heap such that any storage request can be satisfied without having to request more from the operating system. |
|
| Back to top |
|
|
| abhi_thri |
| Posted: Sat Mar 31, 2018 2:19 am Post subject: Re: IIB v10 dataflow engine taking more than 6GB memory. |
|
|
Knight
Joined: 17 Jul 2017
Posts: 516
Location: UK
|
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
