| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Environment Tree memory management |
« View previous topic :: View next topic » |
| Author |
Message
|
| fitzcaraldo |
 Posted: Fri Apr 16, 2004 6:15 pm Post subject: Environment Tree memory management |
 |
|
Voyager
Joined: 05 May 2003
Posts: 98
|
Setup:
WMQI 2.1 CSD 5 AIX 4GB
We have a flow for building file images. It uses a modified version of the MQGet plugin to read up to 10000 messages from a queue which the plugin stores as separate entries in the Environment tree.
Control is then returned to ESQL before another plugin is invoked (java) to combine these records into a single message that is propagated to an output queue. (A file transfer product subsequently reads the message from the queue and writes it to disk)
We currently have a problem in that the flow seems to consume memory (as demonstrated by a declining fre list using vmstat) until it reaches the point where the execution group fails (severe abend - no further info) - after about 3 builds. More specifically, consecutive runs using the same number/size of records, lowers the fre list.
Given that we are writing a large amount of data to the Environment tree, my question is what is the best way to ensure this memory is properly released so it can be reused? On flow completion I have tried setting the relevant Environment entry to NULL and also tried the new delete field function.
eg if we have
Environment.FB.Data.Body[]
set Environment.FB.Data = NULL;
or
delete field Environment.FB.Data;
(I realise there could be a problem in the plugins and that is being investigated.)
Any ideas or suggestions? |
|
| Back to top |
|
 |
| JLRowe |
| Posted: Sat Apr 17, 2004 3:00 am Post subject: |
|
|
Yatiri
Joined: 25 May 2002
Posts: 664
Location: South East London
|
I would suggest that you have reached the limit of what WMQI can do, 10000 is a lot of nodes on a tree!
You can either persevere with WMQI, and perhaps store several messages in a String and reduce the number of nodes, or probably the best idea is to forget WMQI and write it all in Java. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Sat Apr 17, 2004 5:19 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
The thing to keep track of is if you are deleting the various MbNodes that you create in your plugin code. If you've linked them properly into the tree, then you can set them Null or delete them otherwise without destroying the contents.
This will make sure that the reference count on the various elements is correct, so that when you delete or set them to NULL in your ESQL, they will actually get deleted.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| fitzcaraldo |
| Posted: Sat Apr 17, 2004 2:32 pm Post subject: |
|
|
Voyager
Joined: 05 May 2003
Posts: 98
|
Jeff, could you elaborate a bit? What do you mean by 'linking them properly in the tree' and 'the reference count on each element'?
In the plugin, I create a new element in a given location (eg. Environment.FB.Data) using cniCreateElementAsLastChild and this is my root element for further operations.
Beneath this I create initial elements representing the MQMD, the RFH2 and the body of the first message, again using cniCreateElementAsLastChild. For each additional message read I use cniCreateElementAfter to create a new element for the relevant bit (MQMD, RFH2 and Body)
So for 4 messages, I arrive at a structure like this (as shown in a trace dump)
Environment.FB.Data.MQMD
Environment.FB.Data.MQMD
Environment.FB.Data.MQMD
Environment.FB.Data.MQMD
Environment.FB.Data.RFH2
Environment.FB.Data.RHF2
Environment.FB.Data.RFH2
Environment.FB.Data.RHF2
Environment.FB.Data.Body
Environment.FB.Data.Body
Environment.FB.Data.Body
Environment.FB.Data.Body
Now, given that there is no cniDeleteElement plugin API, I have assumed that release of the memory will be handled by the broker on completion of the flow.
Just to make sure though, I have been using the statement
set Environment.FB.Data=NULL
and more recently
delete field Environment.FB.Data
Are you suggesting that I may need to do this for EACH element I created? Is there no implied cascading delete?
So Jonathan, it's not 10000 elements in the tree, it's actually 30000! But it's only memory and if you add it all up, not even that much (the payloads are generally small), so why should it matter - as long as it is correctly released! |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Sat Apr 17, 2004 4:10 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
The basic point is to make sure that the broker can handle it's memory management the way it wants to, and not think that your plug-in code is still using any memory that you've caused to be allocated.
I don't know for sure that it needs to be done for the Environment tree, but you definately need to issue cniFinalize and cniDeleteMessage on any new message trees you create in a plug-in. It may not hurt to do it on the Environment tree you get back from cniGetEnvironmentMessage (or whatever the 2.1 call is, sorry I'm only working with v5 docs right now).
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| fitzcaraldo |
| Posted: Sun Apr 18, 2004 4:43 pm Post subject: |
|
|
Voyager
Joined: 05 May 2003
Posts: 98
|
Ahem...the plugin was leaking.
Thanks for your help. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
