| Author |
Message
|
| gteplits |
 Posted: Wed Jun 25, 2003 12:42 pm Post subject: WMQI abends on multiple propagate |
 |
|
Apprentice
Joined: 16 Jul 2001
Posts: 29
|
We developed a flow, which consists from one Compute node, input and output.
Compute node reads DB2 table into Environment variable array.
For each element of array CWF message is created, which consists from 20 fields. Then message propagates, next element is incremented etc.
We hit limitation of about 500 records, after which broker crushed. I have read in previous discussions about Propagate behaviour that it does not release OutputRoot structure from the memory. What can be done about it?
Any idea is greatly appreciated. |
|
| Back to top |
|
 |
| kirani |
| Posted: Wed Jun 25, 2003 1:01 pm Post subject: |
|
|
Jedi Knight
Joined: 05 Sep 2001
Posts: 3779
Location: Torrance, CA, USA
|
What is the exact error message you are seeing in the event viewer? Can you take the broker level user trace and see if you could find more information there. Will need more information about your message flow or the ESQL code to identify the exact problem.
I don't think this problem is caused by the PROPGATE command. The PROPAGATE command does not create a new copy of the tree each time, instead it clears the output message trees and reclaims the memory ready for further use.
Try to populate the Environment tree from DB and see how many records you are able to retrieve. Try to increse your DB heap size and see if that makes any difference.
_________________
Kiran
IBM Cert. Solution Designer & System Administrator - WBIMB V5
IBM Cert. Solutions Expert - WMQI
IBM Cert. Specialist - WMQI, MQSeries
IBM Cert. Developer - MQSeries
|
|
| Back to top |
|
|
| Craig B |
| Posted: Wed Jun 25, 2003 10:28 pm Post subject: |
|
|
Partisan
Joined: 18 Jun 2003
Posts: 316
Location: UK
|
The previous update was correct in that after each PROPAGATE has completed the storage used by the Output message trees is reclaimed and re-used. However, there was a defect in WMQI V2.1 CSD01 such that this did not happen and this was fixed in CSD02. What CSD level are you running on?
Also what platform is your broker running on? It should be noted that on AIX by default a broker execution group get two memory segments which gives it a hard limit of 512MB. This can be increased if necessary. As you might expect a flow does not have access to the entirety of this 512MB and this is shared between the Execution groups admin side, and all the other message flows. When retrieving a large amount of data from a database then it is possible that you are hitting this hard limit on the Execution group. If you are not running on AIX then this does not apply, however if it is a different Unix platform then you may wish to check your ulimits (soft limits).
From my experience, the typical reasons for an Execution Group abending in the scenario you have described are :
1) A large amount of storage is required which the Execution Group cannot acquire because of system soft and hard limits, or the operating system just does not have the amount of storage available.
2) If a loop is wired in the message flow (ie propagate is not used) the message flows allowable stack size can be exceeded.
3) A database statement is executed within a large loop using the PASSTHRU statement without parameter markers and this causes the prepared statement cache to grow too large.
4) A condition is encountered in the interaction with the database which the message flow/execution group cannot handle. Typically this happens when a remote database is being accessed (or using TCP/IP loopback on AIX such that the DB) and the server database is cancelled/terminated.
From your description it would seem that you would not be encountering (2) or (4). However I would monitor the DataFlowEngine process to which the message flow belongs to see if it is a storage issue or not. Also check your application to make sure you are not using (3).
If you only have one Execution Group and execution group then you just need to locate the DataFlowEngine process on your operating system system usage information. If you have more than one then multiple DataFlowEngine processes will be seen. To locate the correct one, issue the mqsilist command passing your broker name is as the parameter and this will give you a list of your Execution groups. When the broker is running then this list will also display the process Id of each execution group to the right of the execution group name. You can use this process Id to locate the correct DataFlowEngine process.
With the correct DataFlowEngine process identified then you can use the Task Manager on Windows, or ps -ef command on Unix platforms (or USS on zOS) or the S.DA panel on zOS to monitor storage usage. Or obviously any process monitoring tool you prefer to monitor your storage usage. 
If these pointers do not help resolve or identify the problem then could you add some error details to this thread. From a core or abend file a stack trace is generated or can be generated. Would it be possible for you to append this?
I hope this helps.
_________________
Regards
Craig |
|
| Back to top |
|
|
| gteplits |
| Posted: Thu Jun 26, 2003 5:21 am Post subject: |
|
|
Apprentice
Joined: 16 Jul 2001
Posts: 29
|
Kiran, Craig - thank you for your responses.
Our run-time environment:
AIX 5.0
WMQI 2.1 with NO CSD
UDB DB2 7.1
WMQ 5.2 with CSD4
We do have large amount of data returned from SQL statement. That is the reason we are using PROPAGATE - to disect large messages into small pieces for further transformation.
To isolate the problem, I created two Compute nodes. First node performs only database SQL access and places data into Environment array.
Second compute node reads array and perform PROPAGATE. Abend with core happenes during processing in second Compute node.
Currently we can not apply latest CSD (non-technical reasons). I will try to investigate core dump.
How do I increase hard limit of 512 M for execution group?
Do I increase value of MQSI_THREAD_STACK_SIZE? |
|
| Back to top |
|
|
| Craig B |
| Posted: Thu Jun 26, 2003 5:27 am Post subject: |
|
|
Partisan
Joined: 18 Jun 2003
Posts: 316
Location: UK
|
If you have no CSD installed then you will not have the fix for the memory leak on PROPAGATE. Therefore the likely cause for your abend is the number of propagates. Increasing the 512MB hard limit may not help you because if you are encountering a known problem then it is possible (if not likely) you will just keep hitting what hard limit you set. Therefore it might make more sense to go to CSD02 if this is the case.
Since you are using PROPAGATE then it is very unlikely you will be hitting Thread Stack limits.
_________________
Regards
Craig |
|
| Back to top |
|
|
| kegriffi |
| Posted: Thu Jan 22, 2004 10:26 am Post subject: WMQI abends on multiple propagate |
|
|
Newbie
Joined: 22 Jan 2004
Posts: 4
|
| Try putting the code at the top of the node (the code above the line) in your loop after each propagate.. this recopies the headers and gives you a fresh outputRoot.. |
|
| Back to top |
|
|
|
|
