| Author |
Message
|
| scravr |
 Posted: Mon Apr 01, 2013 9:48 am Post subject: File Read Input Node |
 |
|
Partisan
Joined: 03 Apr 2003
Posts: 391
Location: NY NY USA 10021
|
I have a flow triggered by files on system directory.
Any one has idea how to read 1st record ONLY?
I have filez over 2GB and need to read 1st record only. |
|
| Back to top |
|
 |
| lancelotlinc |
| Posted: Mon Apr 01, 2013 9:50 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
Two flows: flow A and flow B. Flow A breaks apart the file into bite-sized chunks. Flow B consumes the chunks.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| scravr |
| Posted: Mon Apr 01, 2013 10:07 am Post subject: |
|
|
Partisan
Joined: 03 Apr 2003
Posts: 391
Location: NY NY USA 10021
|
I need to read 1st record only. NOT the whole file.
File is greater that 2GB
How would you configure flowA RecordDetection? |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Apr 01, 2013 10:12 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
|
| Back to top |
|
|
| scravr |
| Posted: Mon Apr 01, 2013 10:19 am Post subject: |
|
|
Partisan
Joined: 03 Apr 2003
Posts: 391
Location: NY NY USA 10021
|
PLEASE DO NOT ANSWER IF YOU DO NOT READ THE QUESTION !!!
I am not interested in reading all records in file. File may have over 10milion records. reading all is time consuming.
I am looking to read first reaord only, get files name only. |
|
| Back to top |
|
|
| adubya |
| Posted: Mon Apr 01, 2013 10:25 am Post subject: |
|
|
Partisan
Joined: 25 Aug 2011
Posts: 377
Location: GU12, UK
|
The FileRead node allows you to specify a Record Selection Expression which I believe (from reading the Infocenter, I've not used it) can be set to only match the first record i.e. set the expression to:-
| Code: |
| $OutputLocalEnvironment/File/Read/RecordNumber=1 |
Obviously the FileRead node is intended to have an input terminal and not be an initial input node but you could trigger it via a timer node + compute node or similar.
Or if all that's too much grief then use a JCN and access the Java file io library. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Apr 01, 2013 10:31 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| scravr wrote: |
PLEASE DO NOT ANSWER IF YOU DO NOT READ THE QUESTION !!!
I am not interested in reading all records in file. File may have over 10milion records. reading all is time consuming.
I am looking to read first reaord only, get files name only. |
I read the question, and gave you the answer, which is in the InfoCentre. What about my answer is misleading or pointing to the wrong information?
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| scravr |
| Posted: Mon Apr 01, 2013 10:38 am Post subject: |
|
|
Partisan
Joined: 03 Apr 2003
Posts: 391
Location: NY NY USA 10021
|
I have FileInput node that gets error when reading whole file over 2GB.
So I like to read just part of it (or 1st reord only).
No interest in parsing (or spliting by recorsd) and reading 1 rec at a time.
and YES, I read the infocenter many times.
Any ideas? |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Apr 01, 2013 10:39 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| lancelotlinc wrote: |
| I read the question, and gave you the answer, which is in the InfoCentre. What about my answer is misleading or pointing to the wrong information? |
Your answer (and the included link) is how to split a file into records. The question is how, having identified the record layout & format, you only read the first one and not the entire contents of the file.
So neither misleading nor pointing to wrong information, but answering a different question to that asked IMHO.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| adubya |
| Posted: Mon Apr 01, 2013 10:47 am Post subject: |
|
|
Partisan
Joined: 25 Aug 2011
Posts: 377
Location: GU12, UK
|
| scravr wrote: |
| I have FileInput node that gets error when reading whole file over 2GB. |
Not surprised, don't use Whole File mode for this particular requirement.
| Quote: |
So I like to read just part of it (or 1st reord only).
No interest in parsing (or spliting by recorsd) and reading 1 rec at a time.
|
If you want the first record then you must do some spliting. I made a suggestion (above) as to how the FileRead node could be configured to only propagate the first record. Don't you think that will work ?
Yes, see above  |
|
| Back to top |
|
|
| scravr |
| Posted: Mon Apr 01, 2013 10:54 am Post subject: |
|
|
Partisan
Joined: 03 Apr 2003
Posts: 391
Location: NY NY USA 10021
|
| we are talking about FileInput Node |
|
| Back to top |
|
|
| adubya |
| Posted: Mon Apr 01, 2013 10:57 am Post subject: |
|
|
Partisan
Joined: 25 Aug 2011
Posts: 377
Location: GU12, UK
|
| scravr wrote: |
| we are talking about FileInput Node |
I don't think what you want to do is possible with the FileInput node, hence my suggestions around the FileRead node or using a JCN. |
|
| Back to top |
|
|
| zpat |
| Posted: Mon Apr 01, 2013 1:07 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
You say you want to read part of it, without defining what that might be.
What are the record delimiters in the file? |
|
| Back to top |
|
|
| Esa |
| Posted: Mon Apr 01, 2013 11:27 pm Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
Set Record detection so that you can process the file one record at the time, as instructed above.
Set Retry mechanism to Failure (it's the default) and select the best alternative of Action on failing file, for example Add Time Stamp and Move to Backout Subdirectory
After processing the first record, put out the results outside transaction and then just throw an exception. |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Apr 03, 2013 11:24 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
If you only want to get the filename of the file being read, then why do you need to access the contents of the file at all. Doesn't the FileInput node put that information into the LocalEnvironment tree?
If you have some other requirement then please explain exactly what you want your message flow to do. If you are getting an error from the message flow then quote the error message - in full. |
|
| Back to top |
|
|
|
|
