MQSeries.net :: View topic

wbi_telecom · Posted: Thu May 28, 2009 12:18 pm Post subject: Problem with Collector node

Environment :wmb 6.1.0.3 on Z linux

We are using a collector node to create a huge XML message from a batch file. The batch file contains flat records which are read using File input node and converted to XML messages using a compute node and fed to a collector node. The collector node is configured using a timeout value and a message is sent to the Control terminal of the collector node from the End of File terminal of the FileInput node to indicate the end of file.
This design works great for files containing upto 8000 records but fails when the record size goes higher. The memory used is too much for the server (we have 4 GB server).
Another thing is that the message sent to control terminal does not help in getting the collected messages out as it anyway has to wait for either the time or the quantity to complete the collections.
I have following questions

1) Is there a way to pass the Quatity value dynamically to the collector node using Local environement(like we set Queue name or timeout in MQGet)? if that's possible I can get the collection out after all the records in the file are processed and will not rely on timeout or control message.

2) Is there a way to design this process in any other way that will be low on memory. Right now the memory is spent in re formatting the message that comes out of collections which is quite big.

3) Is there a way to get all the messages collected by sending the control message to Collector node? Right now we have the Event Coordination property set to First Complete Collection and the Timeout set to 600 and Quantity 0. Using this configuration we have to wait for 10 minutes for every file even after sending the control message.

Any pointers or suggestions on alternate design are welcome.

Cheers,