| Author |
Message
|
| jrsdm |
 Posted: Tue Nov 06, 2012 3:00 am Post subject: |
 |
|
Apprentice
Joined: 24 Oct 2012
Posts: 27
|
I have changed the file input node record detection property to "Parsed record sequence" and still it is is picking whole file as one.
That;s I have already explained you in the previous post.
The user trace is the outcome of the same flow(where record detection is Parsed record sequence) |
|
| Back to top |
|
 |
| kimbert |
| Posted: Tue Nov 06, 2012 3:27 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
I know you have provided this kind of information before, but...
Please can you post a small version of the XML input document.
Please use [ c o d e] tags to preserve the indentation. There is a button that you can use, just above the box where you type your reply.
Please ensure that the structure of the document that you post is *identical* to the structure of the real file. |
|
| Back to top |
|
|
| jrsdm |
| Posted: Tue Nov 06, 2012 5:30 am Post subject: |
|
|
Apprentice
Joined: 24 Oct 2012
Posts: 27
|
| Code: |
<?xml version="1.0" encoding="UTF-8" ?>
- <pi:Store_Extract xmlns:pi="xyz:com.abd/micof">
- <pi:StoreGroup>
- <pi:Header>
<pi:Version>15</pi:Version>
<pi:Store_ID>123</pi:Store_ID>
<pi:Store_Company_Name>bcd</pi:Store_Company_Name>
<pi:Store_Group_ID>1234</pi:Store_Group_ID>
<pi:Store_Group_Name>abc</pi:Store_Group_Name>
</pi:Header>
- <pi:Department1>
---------------
---------------
</pi:Department1>
- <pi:Department2>
---------------
---------------
</pi:Department2>
- <pi:Department3>
---------------
---------------
</pi:Department3>
</pi:StoreGroup>
</pi:Store_Extract>
|
Above the sample I am using .. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Nov 06, 2012 5:51 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jrsdm wrote: |
| Above the sample I am using .. |
And the parsing settings you selected when you made the modification to use parsed record sequence were.....?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Nov 06, 2012 6:33 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
That sample file contains exactly one XML document. One XML document == one 'record'. That explains why the entire file is being consumed.
Earlier in this thread, you described your input file, and it contained many separate XML documents. I actually drew your attention to this.
Then you told me that your input file was like the SalesList document in the Large Message sample.
Then you withdrew that, and I assumed that you were using the multi-document file format.
I am very, very confused about what you are trying to do. Please stop now and read this entire thread. Make sure that you understand everything that you have been told. Do not post again until you are sure that you are not going to waste our time any more. |
|
| Back to top |
|
|
| jrsdm |
| Posted: Tue Nov 06, 2012 6:34 am Post subject: |
|
|
Apprentice
Joined: 24 Oct 2012
Posts: 27
|
| yes in the record detection I am using parsed record sequence |
|
| Back to top |
|
|
| jrsdm |
| Posted: Tue Nov 06, 2012 6:40 am Post subject: |
|
|
Apprentice
Joined: 24 Oct 2012
Posts: 27
|
| Code: |
I have tried to match my structure with the salelist example of Large messaging sample.
My structure is
-saleEnvelope
-Salelist
-<Header>
</Header>
-<department1>
</department1>
-<department2>
</department2>
-<department3>
</department3>
-<department4>
</department4>
-<department5>
</department5>
</Salelist>
</SaleEnvelope>
|
The previous sample which I have posted was same which I have posted now(below), I am not sure where is the confusion.
| Code: |
<?xml version="1.0" encoding="UTF-8" ?>
- <pi:Store_Extract xmlns:pi="xyz:com.abd/micof">
- <pi:StoreGroup>
- <pi:Header>
<pi:Version>15</pi:Version>
<pi:Store_ID>123</pi:Store_ID>
<pi:Store_Company_Name>bcd</pi:Store_Company_Name>
<pi:Store_Group_ID>1234</pi:Store_Group_ID>
<pi:Store_Group_Name>abc</pi:Store_Group_Name>
</pi:Header>
- <pi:Department1>
---------------
---------------
</pi:Department1>
- <pi:Department2>
---------------
---------------
</pi:Department2>
- <pi:Department3>
---------------
---------------
</pi:Department3>
</pi:StoreGroup>
</pi:Store_Extract>
|
|
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Nov 06, 2012 7:25 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| The previous sample which I have posted was same which I have posted now(below) |
Not true. You previously posted an example that contained a sequence of XML documents with document root tags <CATALOG1>, <CATALOG2>, <CATALOG3> etc. Do you still need to process that file format? Do have you two different file formats that you want to process? |
|
| Back to top |
|
|
| jrsdm |
| Posted: Tue Nov 06, 2012 8:09 am Post subject: |
|
|
Apprentice
Joined: 24 Oct 2012
Posts: 27
|
| Quote: |
When you told me that your input file was like the SalesList document in the Large Message sample.
|
This is only the salelist sample I have posted and I have only this format file(4GB) only which I need to process.
Thanks |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Nov 06, 2012 9:54 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jrsdm wrote: |
| yes in the record detection I am using parsed record sequence |
Used it how? If you just selected ParsedRecordSequence from the Record Detection drop down in the input node, I've found your problem.
I agree with @kimbert - I have no clear idea of what you're trying to do. Unless you restate your problem clearly and in detail, with examples & what you've got configured, further help will be problematic.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jrsdm |
| Posted: Wed Nov 07, 2012 5:14 am Post subject: |
|
|
Apprentice
Joined: 24 Oct 2012
Posts: 27
|
Ok. Here is the problem I am stated again,
1.The file is of 4GB with the structure already posted.
2.Need to process the file record by record so that each department has on file created in the output.
Thanks |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Nov 07, 2012 6:51 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
OK. I'm going to assume that you are referring to the file format that you posted yesterday, and not the <CATALOG1> sample that you used at first. I'm still confused about why you mentioned that format if you do not need to process it.
If my assumption is correct then the Large Message sample contains all the info that you need. Your input document is a single, well-formed XML document. You cannot ask the FileInput node to split up the document into 'records' because the file is a single document. Instead, you will need to write ESQL code ( as shown in the sample ) to process the document in a loop. Each iteration will process one occurrence of the repeating element and then delete the message tree before starting the next one.
So just to be 100% clear, you *cannot* use Parsed Record Sequence for this new input file format. |
|
| Back to top |
|
|
| wbi_telecom |
| Posted: Wed Nov 07, 2012 7:51 am Post subject: |
|
|
Disciple
Joined: 15 Feb 2006
Posts: 188
Location: Harrisburg, PA
|
As far as I remember the large message sample works only if one can read the entire file as a single record in the flow. The problem is a 4 GB file cannot be read as a single message using a file input node which is why jrdsm will have to break it before he starts reading it in his flow.
As I wrote before, We circumvented this limitation by reading the records as BLOB using the end tag of repeating block as custom delimiter. Once we read the record in the flow, we reinstate the end tag and use create parse to build the XML. The difference between the message we process and the message that jrdsm posted is we have the same tag repeating after the header.
Cheers, |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Nov 07, 2012 7:58 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
@wbi_telecom: Thanks - I'd forgotten that detail. The BLOB parser is not a streaming parser, which is why you need to use 'Delimited' on the FileInput node.
| Quote: |
| The difference between the message we process and the message that jrdsm posted is we have the same tag repeating after the header. |
Yes - the OP will need to do something similar to your solution, but with an extra feature to cope with the different tags. |
|
| Back to top |
|
|
| jrsdm |
| Posted: Wed Nov 07, 2012 4:37 pm Post subject: |
|
|
Apprentice
Joined: 24 Oct 2012
Posts: 27
|
Hi Wbi_telecom
I have changed the configuration of the flow as follows
Fileinputnode :
Input_parsing:Blob
Records and Elements: Delimited
Delimiter :Custom delimiter.
Custom delimiter: ??
| Quote: |
| We circumvented this limitation by reading the records as BLOB using the end tag of repeating block as custom delimiter. Once we read the record in the flow, we reinstate the end tag and use create parse to build the XML. |
| Quote: |
Could you please elaborate more on the above statements
Thanks
|
|
|
| Back to top |
|
|
|
|
