Posted: Tue Mar 02, 2010 8:32 pm Post subject: File reader node behavior
Master
Joined: 14 Nov 2008 Posts: 203 Location: charlotte,NC, USA
I believe file reader node will lock the file before reading it and that what is mentioned in the infocenter.
But i faced some problem with the file reader node in multi broker environment
Scenario:
I have 1000 files placed at 1 location. one file reader flow will read the file and put it in the queue. This flow is deployed with multiple instances on multiple broker (4) on the same server. All the brokers have access to the location where files are placed.
File reader node is configured to archive after it is done processing the file. Also "overwrite archive file" option is turned off. That mean it will always add a timestamp to the file name before putting it to archive folder.
When all the brokers (4) are running we saw 1002 messages in the queue but 100 files in the archive folder. That means 2 files got processed by 2 different brokers. Ideally we should receive only 1000 files in the queue and 1000 files in archive folder.
Does not anyone faced this scenario before. Can anyone through some light on the reason for this strange behavior.
We are placing the files on the NFS mount. Does it has something to do with the NFS version?
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum