| Author |
Message
|
| Rambabu |
 Posted: Tue Jun 30, 2015 4:50 am Post subject: Continous %of used memory is growing in WMB 7.0.0.7 |
 |
|
Newbie
Joined: 29 Jun 2015
Posts: 5
|
Hi Friends, Need you help in resolving critical issue in my project.
Details:
---------
We have total 6 flows in this prjct deployed in same EG of maxheapSize=2GB, we are hosting 1 SOAP webservice, subscribing 2 webservices & remaining are DB and MQ protocols.
SOAP Response payload size is approx=10MB, we are getting this in XML tag with base64Format.
At one place we have to parse this base64Format 6MB data to XMLNSC domain.
Broker version: 7.0.0.7
MinHeapSize of EG: 32MB
Maxheapsize of EG: 2GB
No. of instances of flows = 4
Observation:
---------------
We observed %memory used by our EG using top command and ResourceStatistics in MB explorer in which it is continously growing for this EG.
We tried posting 5 messages at a time, each time we posted messages memory usage is growing, we believe it is not using same memory again.
From blogs and forums we got to know that DataflowEngine wont release memory back to OS, but EG should reuse this memory right?
Results:
----------
Here are top commands results for each time we posted messages.
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ SWAP COMMAND
18852 wbrkadm 18 0 2867m 340m 53m S 0.3 4.3 0:16.04 2.5g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 2932m 410m 64m S 54.5 5.1 0:17.68 2.5g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 2952m 442m 69m S 93.7 5.5 0:20.50 2.5g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3097m 573m 69m S 92.3 7.2 0:23.28 2.5g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3084m 643m 69m S 94.7 8.1 0:26.13 2.4g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3116m 686m 76m S 39.9 8.6 0:27.33 2.4g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3116m 686m 76m S 8.6 8.6 0:27.59 2.4g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3133m 702m 77m S 27.6 8.8 0:28.42 2.4g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3367m 1.2g 125m R 96.7 15.6 2:10.73 2.1g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3367m 1.2g 125m S 92.7 15.6 2:12.22 2.1g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3367m 1.2g 125m S 11.0 15.6 2:12.55 2.1g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3367m 1.2g 125m S 3.3 15.6 2:12.65 2.1g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3431m 1.2g 125m S 30.6 16.0 2:13.57 2.1g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3431m 1.2g 125m S 22.9 16.0 2:14.26 2.1g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3431m 1.3g 127m S 76.1 16.2 3:19.47 2.1g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3431m 1.3g 127m S 61.1 16.2 3:21.31 2.1g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3431m 1.3g 127m S 27.2 16.2 3:22.13 2.1g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3431m 1.3g 127m R 62.1 16.2 3:24.00 2.1g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3431m 1.3g 127m S 7.3 16.2 3:24.22 2.1g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
18852 wbrkadm 18 0 3495m 1.3g 127m S 38.9 16.4 3:25.39 2.1g DataFlowEngine QL1002S1.BRKR 7c781608-4c01-0000-0080-8319345fd7a6 mb_NGP
workarounds we performed:
---------------------------------
1. we have gone through the code and deleted Environment varibales at end of the flow.
2. Decreased no. of parsing places
3. Deployed flows in different EGs and observed the stats, where we saw that below flow is taking more memory than others.
MQ->Compute->SOAPRequest->Compute->SOAPRequest
(6MB) (response=1MB) (request contains total=10MB)
4. when we restart the EG memory usage is getting reduced to normal.
Kindly help us to resolve this issue, let me know if you need any more info  |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Tue Jun 30, 2015 5:45 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
There are several ways that ESQL and Java code can leave memory behind when they are finished executing.
I would suggest you open a PMR, but whoops you're using v7. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Jun 30, 2015 6:00 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| mqjeff wrote: |
There are several ways that ESQL and Java code can leave memory behind when they are finished executing.
I would suggest you open a PMR, but whoops you're using v7. |
He's lucky (just) V7 is still supported until September.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Jun 30, 2015 8:37 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5849
Location: UK
|
|
| Back to top |
|
|
| ruimadaleno |
| Posted: Wed Jul 01, 2015 12:45 am Post subject: |
|
|
Master
Joined: 08 May 2014
Posts: 274
|
It would be very nice if there were a tool that scan's the dataflowengine (DFE) heap (memory used by dataflowengine - the process "running" the execution group and generate a map or some statistics about memory blocks and objects stored on those memory blocks. Also the abilitity to gather info about empty/non continuous blocks would be very helpful to check if memory fragmentation level has reached a plateau or, if it is time to a execution group restart in order to "clean up" the memory used by execution groups ...
well... sorry for the off topic here  maybe time to a RFE 
_________________
Best regards
Rui Madaleno |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Jul 01, 2015 4:09 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
And remember if you are using JCN nodes that each node that is a create or modify message type needs to have at the end a
| Code: |
} finally{
message.clearMessage();
}
|
If this little snippet is not present you will be leaking native memory.
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
