MQSeries.net :: View topic - File read Node not moving files to transit

MQSeries.net

Tech Exchange

Education

Certifications

Library

Info Center

SupportPacs

FAQÂ Â

Usergroups

RSS Feed - WebSphere MQ Support

RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » File read Node not moving files to transit

File read Node not moving files to transit

« View previous topic :: View next topic »

Author

Message

matuwe

Posted: Mon Nov 11, 2013 8:38 am Post subject: File read Node not moving files to transit

Master

Joined: 05 Dec 2007
Posts: 296

Hi

Has anyone experienced this problem.. I have two brokers reading from the same GPFS file system. If I receive a big file, then while the one broker reads the file, the other broker starts to process it.

Is there a way of locking the file or moving it to transit as the Fileinput does?

2. Why does the file read node read the file even when another process is still writing to the file? Is there a way of preventing this?

I am using broker 7.0.3 Thanks

mqjeff

Posted: Mon Nov 11, 2013 8:52 am Post subject:

Grand Master

Joined: 25 Jun 2008
Posts: 17447

Reading a file is an example of the Halting Problem, in the general case.

You only know that the file is "ready" when you know that the program writing the file has halted.

You should make sure that your fileRead node uses the full file name, not a pattern. Then you can prevent two fileread nodes from reading the same file at the same time.

smdavies99

Posted: Mon Nov 11, 2013 9:17 am Post subject:

Jedi Council

Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.

LArge files take time to write.
Other posts on this topic have suggested the following

1) Write a the file into a directory that is NOT the same one as where it will be read from but preferably on the same filesystem.
2) When the file writing is complete rename/move the file to the directory where the file will be read from.

the rename/move operation is far quicker than writing it when there is a lot of data in the file.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995

Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions.

matuwe

Posted: Mon Nov 11, 2013 12:53 pm Post subject:

Master

Joined: 05 Dec 2007
Posts: 296

Thanks for the info.. Will it be good design if I do this in Java compute .. I only know the directory I need to read from at runtime, from reading the database.. I will use java to read all filenames from the directory. then I can use the arraylist to get separate files using the file Input.

Or is there a way in ESQL to get list of files from a file system?

Display posts from previous:

Page 1 of 1

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » File read Node not moving files to transit

Jump to:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Protected by Anti-Spam ACP