| Author |
Message
|
| alastair |
 Posted: Wed Feb 08, 2012 8:44 pm Post subject: MRM Memory issue: Multiple invokations of MRM parser |
 |
|
Novice
Joined: 08 Feb 2012
Posts: 18
Location: Sydney
|
Hi people. I have a question regarding repetitive MRM parsing within a loop and it's affect on DFE memory consumption. Why does the MRM parser appear to not release memory when called multiple times even after the target trees are deleted?
-------------------------
We have a SOAP response from a provider system which is an (sometimes large) XML payload containing and array of results (up to 2500), each containing two copybook hex blobs. In our transformation ESQL we are looping through the XML result set and using the following statement to parse the BLOB riskData to ROW variable rowRef for processing:
CREATE LASTCHILD OF rowRef
DOMAIN('MRM')
PARSE(riskData
OPTIONS RootBitStream
ENCODING 785
CCSID 500
SET 'polisy.claim.cobol.model'
TYPE msgType
FORMAT 'Binary1');
Once I have finished using the ROW rowRef I issue a:
DELETE FIELD rowRef.[>];
and then iterate the loop and continue to process the next result.
-----------------------
I have tested and tweaked my code extensively to try to understand what was causing the excessive memory consumption and found the the MRM parsing to be the problem. Taking out one of the MRM parses and replacing with a hack (CAST as char, SUBSTRING) reduced the overall memory consumption by half with the same result set.
If anyone is able to shed some light on this behaviour of the MRM I would be most grateful. |
|
| Back to top |
|
 |
| marko.pitkanen |
| Posted: Wed Feb 08, 2012 10:51 pm Post subject: |
|
|
Chevalier
Joined: 23 Jul 2008
Posts: 440
Location: Jamsa, Finland
|
Hi,
I think that MRM like other parsers that parses data to structured data trees uses some meta data assigned to every entity they holds parsed in the tree. That increases memory consumption.
I have myth that Message Broker's memory management strategy is that it keeps every memory blog it ever reserves because it is very likely that the same kind of message arrives in the future and then all the necessary memory is already reserved.
Have you read and tried about possibility to parse / process messages in parts?
--
Marko |
|
| Back to top |
|
|
| alastair |
| Posted: Thu Feb 09, 2012 12:10 am Post subject: |
|
|
Novice
Joined: 08 Feb 2012
Posts: 18
Location: Sydney
|
Thanks for the reply Marco.
| Quote: |
| I think that MRM like other parsers that parses data to structured data trees uses some meta data assigned to every entity they holds parsed in the tree. That increases memory consumption. |
I understand that the DFEs hold on to allocated memory. I don't have a problem with that.
| Quote: |
| I think that MRM like other parsers that parses data to structured data trees uses some meta data assigned to every entity they holds parsed in the tree. That increases memory consumption. |
So you mean that the MRM is consuming memory that cannot be reused within the COMPUTE node? This may be what I am observing. What I am puzzled about, is I am deleting the MRM trees I have created after each iteration, but it makes little difference to the amount of memory consumed whether I delete the MRM trees or not. |
|
| Back to top |
|
|
| marko.pitkanen |
| Posted: Thu Feb 09, 2012 1:53 am Post subject: |
|
|
Chevalier
Joined: 23 Jul 2008
Posts: 440
Location: Jamsa, Finland
|
Hi,
Just guessing but have you double checked that clause:
| Quote: |
| DELETE FIELD rowRef.[>]; |
does the thing you want to?
--
Marko |
|
| Back to top |
|
|
| alastair |
| Posted: Thu Feb 09, 2012 1:59 am Post subject: |
|
|
Novice
Joined: 08 Feb 2012
Posts: 18
Location: Sydney
|
Hi Marco
| Quote: |
| Just guessing but have you double checked that clause |
I have indeed. It deletes the tree from the first (and only) child of rowRef. In this case it is MRM.*[]
I checked it in the debugger and it is working as expected. Just the ROW at rowRef is left behind, ready for the next iteration and parse.
It seems to be the same as:
SET rowRef.[>] = NULL; |
|
| Back to top |
|
|
| marko.pitkanen |
| Posted: Thu Feb 09, 2012 2:08 am Post subject: |
|
|
Chevalier
Joined: 23 Jul 2008
Posts: 440
Location: Jamsa, Finland
|
Hi,
How are you using the output data tree (I mean how are you using the content of the parsed data)?
And can you describe the version and OS of your environment?
--
Marko |
|
| Back to top |
|
|
| alastair |
| Posted: Thu Feb 09, 2012 2:25 am Post subject: |
|
|
Novice
Joined: 08 Feb 2012
Posts: 18
Location: Sydney
|
Hi Marco
| Quote: |
| And can you describe the version and OS of your environment? |
Getting the same issues in our Production Z-Linux 64bit WMB 7.0.0.3 and on local workstation Windows 7 64bit WMB 7.0.0.3.
| Quote: |
| How are you using the output data tree (I mean how are you using the content of the parsed data)? |
Once we've done the BLOB MRM PARSE to rowRef, we then reference the data as rowRef.MRM.FOO.BAR
This works just fine. Once I've done with the content of rowRef, I try to discard the MRM tree to free up the memory for the next iteration.
I've run this locally on an EG containing a single flow and message set which starts at around 120MB. Run a 2500 response payload (containing 5000 hex BLOBs) at it and the EG swells to 390MB. This is in just one compute node. Take out one of the MRM parses and replace with a hack (which avoids 2500 parses, leaving 2500) and we're down to 170MB. |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu Feb 09, 2012 2:43 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
This is a fair question - I don't think you are the first to encounter this problem. I'll do some investigation and get back to you.
Last edited by kimbert on Thu Feb 09, 2012 3:03 am; edited 1 time in total |
|
| Back to top |
|
|
| alastair |
| Posted: Thu Feb 09, 2012 2:46 am Post subject: |
|
|
Novice
Joined: 08 Feb 2012
Posts: 18
Location: Sydney
|
Thanks @kimbert
| Quote: |
| This is a fair question - I don't you are the first to encounter this problem. I'll do some investigation and get back to you. |
Z196 memory is expensive  |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu Feb 09, 2012 5:38 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| alastair wrote: |
Thanks @kimbert
| Quote: |
| This is a fair question - I don't you are the first to encounter this problem. I'll do some investigation and get back to you. |
Z196 memory is expensive |
RHEL Linux memory is not.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| kimbert |
| Posted: Fri Feb 10, 2012 6:25 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Try setting environment variable MQSI_FREE_MASTER_PARSERS to "1". I have reason to believe that it will fix your problem. |
|
| Back to top |
|
|
| alastair |
| Posted: Sat Feb 11, 2012 9:38 pm Post subject: |
|
|
Novice
Joined: 08 Feb 2012
Posts: 18
Location: Sydney
|
Thanks @kimbert
Did as you said and the end result using the same test resulted in a DFE of 135MB. Nice work dude! 
I would like to understand what this setting is doing.
During the test I did observe the DFE grow to over 300MB before shrinking down to 135MB. It seems that the parser is still growing quite large during processing and only releasing memory at node/flow termination.
There is still a question of why is the MRM parser growing at every PARSE invocation instead of recycling memory it used during the previous PARSE.
Could you kindly advise what the possible side-effects of this setting could be? Our DFE in prod has a lot of traffic and was wondering the implications for performance.
On the bright side, this experience will definitely steer our future design for message payloads requiring parsing by the MRM.
Many thanks for your help.
Alastair |
|
| Back to top |
|
|
| zpat |
| Posted: Sun Feb 12, 2012 12:05 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
We have a lot of memory being used by WMB as well - is this environment variable setting generally recommended (and risk free)?
Any reason why it is not enabled by default?
P-series memory from IBM appears to be made from solid gold judging by the prices! |
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Feb 13, 2012 2:32 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| is this environment variable setting generally recommended |
No
The only risk is that it might lower performance without lowering memory consumption.
| Quote: |
| Any reason why it is not enabled by default? |
Yes. The default setting is better for performance, and very few users need the behavior that this environment variable provides. You should not assume that your memory usage will drop just because you set this variable. You should not experiment with setting it unless you have good reason to believe that memory is *leaking*.
In this case, the combination of
a) using a CREATE statement
b) with a PARSE clause
c) in a loop
was the thing that made my ears prick up. |
|
| Back to top |
|
|
| alastair |
| Posted: Mon Feb 13, 2012 2:44 am Post subject: |
|
|
Novice
Joined: 08 Feb 2012
Posts: 18
Location: Sydney
|
Thanks kimbert.
Have you managed to recreate the memory leak using the PARSE inside a loop or was that an educated assumption? If not, then I would be happy to help by providing code/data to assist. Any plans to address this issue in the future?
Alastair |
|
| Back to top |
|
|
|
|
