| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| documentation |
« View previous topic :: View next topic » |
| Author |
Message
|
| enna |
 Posted: Mon Feb 09, 2009 1:02 pm Post subject: documentation |
 |
|
Newbie
Joined: 04 Feb 2009
Posts: 3
|
Is there a separate manual for FTE ?
I would like to to find out if I can start processing the incoming file before it has been completely arrived. I can then overlap sending and receiving processing for large files (10G or more) ? I undersatnd there will be integrety problem if the file transfer never completed succesfully but that is another. issue.
Thx |
|
| Back to top |
|
 |
| RichCumbers |
| Posted: Mon Feb 09, 2009 1:29 pm Post subject: |
|
|
Novice
Joined: 22 May 2006
Posts: 10
Location: Hursley, UK
|
|
| Back to top |
|
|
| enna |
| Posted: Mon Feb 09, 2009 2:47 pm Post subject: |
|
|
Newbie
Joined: 04 Feb 2009
Posts: 3
|
Thanks Rich
However I could not find from that info centre centre whether I can start processing the file before it is completely arrived.
Thanks |
|
| Back to top |
|
|
| RichCumbers |
| Posted: Tue Feb 10, 2009 12:17 am Post subject: |
|
|
Novice
Joined: 22 May 2006
Posts: 10
Location: Hursley, UK
|
| enna wrote: |
However I could not find from that info centre centre whether I can start processing the file before it is completely arrived.
Thanks |
Hi Enna,
We have not done any testing on this functionality.
Knowing what I know about the internals, we do lock files when reading/writing them with the FTE Agent. If your processing is allowed to read the file as it is being written then we cannot stop you from processing the file before the transfer has completed.
We do not guarantee the integrity of the files when this happens, we simply cannot support this mode of operation, as we are not in control of your process and cannot ensure data integrity.
Cheers
Rich |
|
| Back to top |
|
|
| PeteVerdon |
| Posted: Wed Feb 11, 2009 10:19 am Post subject: |
|
|
Apprentice
Joined: 11 Feb 2009
Posts: 33
Location: Hursley, UK
|
Note that when FTE writes a file at the receiving end, it writes into a temporary file and then renames it once complete. So if you're copying myBigFile.txt it will fill up myBigFile.txt.part, and then rename to myBigFile.txt after writing the final chunk of data. This is to make it easier to arrange subsequent processing to start when it sees myBigFile.txt arrive. I'm not aware of a way to turn it off (though I might have missed it).
You'd need to be aware of this rename if you intended to start processing before the transfer finishes. I am not a file I/O expert, but I would expect it to work OK on a Linux filesystem, where your program would continue to have access to the inode regardless of the file's current name. I believe this would be similar on other unices, but I have no idea what would happen on Windows (possibly you couldn't even start reading the file at all?). You'd have to be careful if your program closed and reopened its input file though.
Note that this kind of scenario is firmly outside the way FTE is intended to be used. There's no guarantee that it will work properly now (even if it appears to in simple tests) or that it will continue to work in future.
Pete |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
