Posted: Fri Sep 25, 2009 7:00 am Post subject: Processing large files using FileInput node
Disciple
Joined: 15 Feb 2006 Posts: 188 Location: Harrisburg, PA
Environment wmb 6.1.0.3 on Z Linux.
We are using Fielinput nodes to process large XML files. The flow breaks the file into smaller messages and drops them on a queue.
We process a file as big as 51 MB in 1 minute. But when we test with a file which is 200 MB in size its taking 1 hour. We are trying to figure out why the same code is performing so poorly with 250 mb file. We added more RAM on the server and tried increasing the heap size of the EG but it has no effect on the processing time.
Do you know if we are hitting a upper limit on some resources because of which the flow behaves differently for 250 MB file?
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum