| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Fileinput node question |
« View previous topic :: View next topic » |
| Author |
Message
|
| assubct |
 Posted: Wed May 14, 2008 6:40 am Post subject: Fileinput node question |
 |
|
Newbie
Joined: 15 Sep 2004
Posts: 8
|
Hi all,
I need to process files in a logical order.
Someone know what's the order of file processing on fileinput node?
Is it based on file name or in file timestamp?
I'll appreciate any ideas on how to solve that.
Thanks! |
|
| Back to top |
|
 |
| Gaya3 |
| Posted: Wed May 14, 2008 8:05 pm Post subject: Re: Fileinput node question |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
| assubct wrote: |
I need to process files in a logical order.
Someone know what's the order of file processing on fileinput node?
Is it based on file name or in file timestamp?
|
you can specify file name patterns, as its supported by MB V6.1
say gaya*.cfg, where it process the files with cfg suffix and preffix of gaya. if it doesnt meet the criteria, it wont touch other files.
Regards
Gayathri
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| AkankshA |
| Posted: Wed May 14, 2008 8:17 pm Post subject: |
|
|
Grand Master
Joined: 12 Jan 2006
Posts: 1494
Location: Singapore
|
sequencing of processing for files or any specific order is not maintained by FileInput node... it simply picks them and put them in an internal directory and processes them..
_________________
Cheers |
|
| Back to top |
|
|
| mqpaul |
| Posted: Tue May 27, 2008 5:53 am Post subject: FWIW, the FileInput node processes files by last modified |
|
|
Acolyte
Joined: 14 Jan 2008
Posts: 66
Location: Hursley, UK
|
The FileInput node scans the input directory for file names matching the pattern property. It sorts the results in order of last modified time stamp, and starts processing the oldest first.
This information does not help you design flows, since the sequence will usually be disrupted. For example, if you have several instances of your input node, several files get processed in parallel. Early files may take more processing than later ones, so the first to start may not be the first to finish.
Even if there is only one instance, you still can't trust the processing sequence: while the FileInput node is processing a file, other processes may move files with older last modified dates into the input directory, and they will be processed on the next scan.
If you need to process file contents in some particular sequence, I think you will need to store them on intermediate storage (such as a database table, or write them all to one file), sort that into the desired sequence, and use the results in your flow.
_________________
Paul |
|
| Back to top |
|
|
| yaakovd |
| Posted: Tue May 27, 2008 7:03 am Post subject: |
|
|
Partisan
Joined: 20 Jan 2003
Posts: 319
Location: Israel
|
You can separate file reading from file processing (2 message flows).
Flow 1:
1. read files using fileinput node
2. set some MQMD field (e.g. priority) by your file name
3. send files to queue
3. send control message to other queue
Flow 2:
1. read control message
2. MQ Get by predefined MQMD field.
Other option - OS script (perl or other) that renames files according your logic to: {order sequence}.{orig name}.{MB input suffix}
_________________
Best regards.
Yaakov
SWG, IBM Commerce, Israel |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
