| Author |
Message
|
| dogorsy |
 Posted: Tue Aug 06, 2013 7:41 am Post subject: |
 |
|
Knight
Joined: 13 Mar 2013
Posts: 553
Location: Home Office
|
| dogorsy wrote: |
| mqjeff wrote: |
It's recommended practice to construct loops in good ways.
You can't just say "don't use loops". that's like saying "Don't use integers". |
or even: "use loops with integer counters, but do not increment them"  |
Anyway, it is all academic, as the OP has not bothered to answer. I am sure you all know how to code a loop to exercise a file read node and retrieve all records, not so sure about the raiser. |
|
| Back to top |
|
 |
| visasimbu |
| Posted: Mon Aug 12, 2013 1:44 pm Post subject: Re: File Read Node record number update |
|
|
Disciple
Joined: 06 Nov 2009
Posts: 171
|
Apologies for late reply!
I am not using a loop to get the records.
I have file read node which will read a file by chunk. It means reading (chunk by chunk)5MB of bytes from 250MB of file. By setting below properties in file read node
Under Records and Elements.
Record detection : Fixed lenght
Lenght : 5242880
There is no loop to fetch each 5MB data. It will automatically fetches next 5MB of data once the previews 5MB got processed in the file.
Actually for FileInput Node , we have below properties which will intimate whether the record is first record
| Code: |
IF InputLocalEnvironment.File.Record = 1
THEN
SET OutputRoot.XMLNSC.Dept.(XMLNSC.Attribute)Date = CURRENT_DATE;
END IF; |
So, I am checking any property from fileRead node will intimate the record is first record.
But I could not find any property like that.
So I have resolved it by declaring and incrementing a integer (in compute node2) by manually. |
|
| Back to top |
|
|
| kash3338 |
| Posted: Mon Aug 12, 2013 7:52 pm Post subject: Re: File Read Node record number update |
|
|
Shaman
Joined: 08 Feb 2009
Posts: 709
Location: Chennai, India
|
| visasimbu wrote: |
| There is no loop to fetch each 5MB data. It will automatically fetches next 5MB of data once the previews 5MB got processed in the file. |
How do you achieve this in FileRead node with out loop's? AFAIK FileRead node reads only one record and using the property "Record Selection Expression" you can specify which record you want to read from the file. By default, this property is set to true which will give you the first record always and hence the value of "LocalEnvironment.File.Read.RecordNumber" is always "1" for you.
But still I dont think it is possible to read all the records from the file by the way you have mentioned unless you use a loop in the flow (say for example a PROPAGATE before the FileRead node). |
|
| Back to top |
|
|
| visasimbu |
| Posted: Tue Sep 03, 2013 6:26 am Post subject: Re: File Read Node record number update |
|
|
Disciple
Joined: 06 Nov 2009
Posts: 171
|
| kash3338 wrote: |
But still I dont think it is possible to read all the records from the file by the way you have mentioned unless you use a loop in the flow (say for example a PROPAGATE before the FileRead node). |
Kashyap,
I have achived it by below steps.
Properties of file Read node -> Records and Elements -> Record Detection->Fixed Length.
Lenght-> NoOfBytes. (No of bytes I like to read as a record).
Hence it will read the data by bytes as a record without any loop in the flow. We can achieve same in the fileInput node as well. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Sep 03, 2013 7:38 am Post subject: Re: File Read Node record number update |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| visasimbu wrote: |
| kash3338 wrote: |
But still I dont think it is possible to read all the records from the file by the way you have mentioned unless you use a loop in the flow (say for example a PROPAGATE before the FileRead node). |
Kashyap,
I have achived it by below steps.
Properties of file Read node -> Records and Elements -> Record Detection->Fixed Length.
Lenght-> NoOfBytes. (No of bytes I like to read as a record).
Hence it will read the data by bytes as a record without any loop in the flow. We can achieve same in the fileInput node as well. |
That reads a single record, or more accurately a single block of bytes.
It does not read more than one block of bytes. |
|
| Back to top |
|
|
| dogorsy |
| Posted: Tue Sep 03, 2013 11:24 pm Post subject: Re: File Read Node record number update |
|
|
Knight
Joined: 13 Mar 2013
Posts: 553
Location: Home Office
|
| visasimbu wrote: |
I have achived it by below steps.
Properties of file Read node -> Records and Elements -> Record Detection->Fixed Length.
Lenght-> NoOfBytes. (No of bytes I like to read as a record).
Hence it will read the data by bytes as a record without any loop in the flow. We can achieve same in the fileInput node as well. |
So you are only getting one chunk of data, with the length NoOfBytes, YOU ARE NOT READING ALL THE RECORDS, and that is why I was asking how did you manage to read all the records. |
|
| Back to top |
|
|
| paustin_ours |
| Posted: Sun Mar 02, 2014 8:32 am Post subject: |
|
|
Yatiri
Joined: 19 May 2004
Posts: 667
Location: columbus,oh
|
Fileread only reads one record. In my case i am using a loop in a preceding compute node to propagate to get consecutive records from file and sending each record out as an individual output.
you guys have mentioned that there is an alternative to looping? Can you pls shed some light on that? Can't think of any other way besides looping. |
|
| Back to top |
|
|
| dogorsy |
| Posted: Sun Mar 02, 2014 9:54 am Post subject: |
|
|
Knight
Joined: 13 Mar 2013
Posts: 553
Location: Home Office
|
| paustin_ours wrote: |
Fileread only reads one record.
|
are you sure ?!
| Quote: |
you guys have mentioned that there is an alternative to looping? |
A good alternative would be to read the fileRead node parameters and possible values.
Last edited by dogorsy on Sun Mar 02, 2014 11:07 pm; edited 1 time in total |
|
| Back to top |
|
|
| paustin_ours |
| Posted: Sun Mar 02, 2014 11:55 am Post subject: |
|
|
Yatiri
Joined: 19 May 2004
Posts: 667
Location: columbus,oh
|
Yes I am sure. And forgive me, not sure what you are trying to say in your
Second statement. I am splitting a file into individual records using fileread
. This I am doing through a loop and propagate. Wanted to see if there was a
Better way. |
|
| Back to top |
|
|
| dogorsy |
| Posted: Sun Mar 02, 2014 10:54 pm Post subject: |
|
|
Knight
Joined: 13 Mar 2013
Posts: 553
Location: Home Office
|
| paustin_ours wrote: |
| Yes I am sure. |
well, you are wrong
| Quote: |
And forgive me, not sure what you are trying to say in your Second statement. |
I am trying to say the the fileRead node is documented in the infocentre. If you read the properties, you will find there is one that gives you alternatives on how to read a file. As to whether there is a better way, that is for you to decide, depending on file size, format, etc. |
|
| Back to top |
|
|
| paustin_ours |
| Posted: Mon Mar 03, 2014 4:52 am Post subject: |
|
|
Yatiri
Joined: 19 May 2004
Posts: 667
Location: columbus,oh
|
| well i don't think i am wrong. I have tested it. When i set the fileread to look for a record that is delimited, It finds it and send it to the output. It doesnt go back and send me consecutive records it finds and outputs them as well until the end of the file. |
|
| Back to top |
|
|
| paustin_ours |
| Posted: Mon Mar 03, 2014 5:03 am Post subject: |
|
|
Yatiri
Joined: 19 May 2004
Posts: 667
Location: columbus,oh
|
| I also looked at the infocenter and there is no setting that would allow that. I hope you know what you are talking about. |
|
| Back to top |
|
|
| dogorsy |
| Posted: Mon Mar 03, 2014 5:48 am Post subject: |
|
|
Knight
Joined: 13 Mar 2013
Posts: 553
Location: Home Office
|
| paustin_ours wrote: |
| I also looked at the infocenter and there is no setting that would allow that. I hope you know what you are talking about. |
Yes, I know what I am talking about. If you set the right parameter, you can read the whole file, which means what you said before
| Quote: |
Fileread only reads one record
|
is wrong, and YOU don't know what you are talking about. You control how to read the file by setting different options, so, go back and read again. |
|
| Back to top |
|
|
| paustin_ours |
| Posted: Mon Mar 03, 2014 6:27 am Post subject: |
|
|
Yatiri
Joined: 19 May 2004
Posts: 667
Location: columbus,oh
|
| I suggest you read the question asked before jumping in with useless comments and answers. I wasn't even asking about reading the whole file. |
|
| Back to top |
|
|
| dogorsy |
| Posted: Mon Mar 03, 2014 6:37 am Post subject: |
|
|
Knight
Joined: 13 Mar 2013
Posts: 553
Location: Home Office
|
| paustin_ours wrote: |
| I suggest you read the question asked before jumping in with useless comments and answers. I wasn't even asking about reading the whole file. |
you asked for alternatives, and you said the fileRead node reads one record at a time, WHICH IS WRONG, only if you configure it that way.
So before telling me about useless comments, please learn to express yourself in English, not only ignorant but also arrogant you are. |
|
| Back to top |
|
|
|
|
