Posted: Thu May 13, 2010 7:06 am Post subject: FTP transfer of files - FileOutput Node ERROR
Centurion
Joined: 18 Nov 2009 Posts: 118
Hi All,
We have a problem with file output nodes.
We get a huge file through file input node. We do computing on that file and generate severaly small files. Now as these files have to be transferred to another FTP server.
Since the number of output files generally are too many and while in the process of transfer of these files if there is a network error, the tranfer of files stops, an exception in the broker is generated for the input file.
Now my problem is while reprocessing. Few number of files have already recahed the destination FTP server and has got processed. Is there any way we can ensure that all the outputs get generated locally and they get transferred and if there is any network error, I can reprocess them again.
Joined: 10 Feb 2003 Posts: 6076 Location: Somewhere over the Rainbow this side of Never-never land.
How about splitting the operation into two flows.
The first reads the message and splits it up and puts them onto a queue
The (new) second one reads each message as a transaction in its own right and is coded to handle errors. Multiple instances would stop blocking in the event of one doing a retry.
Just an idea to start with. _________________ WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions.
Joined: 14 Apr 2010 Posts: 522 Location: Craggy Island
You could untick the remote transfer box so your flow creates all the files locally and doesn't automatically do the transfer. Then when the flow finishes successfully, ftp the files to their destinations. _________________ Never let the facts get in the way of a good theory.
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum