| Author |
Message
|
| mqbrks |
 Posted: Tue Feb 28, 2017 11:29 am Post subject: Parsing a file - DFDL |
 |
|
Voyager
Joined: 17 Jan 2012
Posts: 75
|
Hi All,
I have a file which has 1 audit record and multiple data records(category1(min occurs 1),category2(min occurs 1),,category3(unbounded))
How to describe the data in DFDL, so that it intelligently takes multiple category 3 records ?
Sample File Contents below
LOGKOING 80TD 1 PR00D 16200008
0113TD1KRE 1 KIMH 1 1 87888FTM -UKK3
0213TD1KRE 1 KIMH 1 1 87888FTM -UKK3
0313TD1KRE 1 KIMH 1 1 87888FTM -UKK3 %HIKK
0313TD1KRE 1 KIMH 1 1 87888FTM -UKK3 $JKHKH
0113TD1KRE 1 KIMH 1 1 87888FTM -UKK3
0213TD1KRE 1 KIMH 1 1 87888FTM -UKK3
0313TD1KRE 1 KIMH 1 1 87888FTM -UKK3 %HIKK
0313TD1KRE 1 KIMH 1 1 87888FTM -UKK3 $JKHKH
LOG - Audit record
011 - category 1
021 - Category 2
031
032 - Category 3 Records
How do I logically define or tell DFDL to parse intelligently based on data to consider the category records? |
|
| Back to top |
|
 |
| timber |
| Posted: Tue Feb 28, 2017 1:02 pm Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
Good problem description. You did not tell us what you have tried, and what happened. Although I'm guessing that the DFDL parser is attempting to parse everything after Category 2 as a Category3 because it doesn't know when to stop.
The answer is to use a discriminator or an assert to guide the DFDL parser down the right path. You specify an XPath expression which gets evaluated against the parsed data. If it evaluates to false then the parser backs out of that branch ( i.e. Category 3 ) and tries the next possibility ( probably the Category 1 of the next record ).
If you use a discriminator, you get an extra feature. If the XPath evaluates to true then the parser will assume that the current branch is the correct one and will not back out of it. Any parsing errors after that ( within the scope of the same point of uncertainty ) will be treated as an error. |
|
| Back to top |
|
|
| mqbrks |
| Posted: Tue Feb 28, 2017 1:29 pm Post subject: |
|
|
Voyager
Joined: 17 Jan 2012
Posts: 75
|
| timber wrote: |
| Good problem description. You did not tell us what you have tried, and what happened. Although I'm guessing that the DFDL parser is attempting to parse everything after Category 2 as a Category3 because it doesn't know when to stop. |
That's exactly right.
| Quote: |
The answer is to use a discriminator or an assert to guide the DFDL parser down the right path. You specify an XPath expression which gets evaluated against the parsed data. If it evaluates to false then the parser backs out of that branch ( i.e. Category 3 ) and tries the next possibility ( probably the Category 1 of the next record ).
|
Just now even I got to know that I have to use discriminator or assert to parse the records. Trying to figure out how to use discriminators, do you have any samples ?
Last edited by mqbrks on Tue Feb 28, 2017 1:34 pm; edited 1 time in total |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Feb 28, 2017 1:31 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| mqbrks wrote: |
| timber wrote: |
| Good problem description. You did not tell us what you have tried, and what happened. Although I'm guessing that the DFDL parser is attempting to parse everything after Category 2 as a Category3 because it doesn't know when to stop. |
That's exactly right. |
He usually is.
| mqbrks wrote: |
| Just now even I got to know that I have to use discriminator or assert to parse the records. Trying to figure out how to discriminators, do you have any samples ? |
| mqbrks wrote: |
LOG - Audit record
011 - category 1
021 - Category 2
031
032 - Category 3 Records |
Among other things, your message seems to be based on a COBOL copybook.
Perhaps you should use that to create your DFDL model...
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| timber |
| Posted: Wed Mar 01, 2017 2:44 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| Quote: |
your message seems to be based on a COBOL copybook.
Perhaps you should use that to create your DFDL model... |
That's a good suggestion. It's worth pointing out, though, that the COBOL importer will not automatically create the asserts/discriminators that are required to guide the parser. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Mar 01, 2017 5:04 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| timber wrote: |
| Quote: |
your message seems to be based on a COBOL copybook.
Perhaps you should use that to create your DFDL model... |
That's a good suggestion. It's worth pointing out, though, that the COBOL importer will not automatically create the asserts/discriminators that are required to guide the parser. |
Well, ok. It'd be nice if it did, but I suppose I have to harangue someone else about that these days.
It does at least build the structures that tell you where to put discriminators/assertions. You might have to add a choice structure though.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| timber |
| Posted: Wed Mar 01, 2017 5:30 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| In general it cannot - a COBOL copybook does not contain the information that the parser requires. If writing a COBOL program to process it, that's program logic added by somebody who understands the data. If designing a DFDL schema, the equivalent rules need to be added as asserts/discriminators. |
|
| Back to top |
|
|
| mqbrks |
| Posted: Wed Mar 01, 2017 1:06 pm Post subject: |
|
|
Voyager
Joined: 17 Jan 2012
Posts: 75
|
| timber wrote: |
| In general it cannot - a COBOL copybook does not contain the information that the parser requires. If writing a COBOL program to process it, that's program logic added by somebody who understands the data. If designing a DFDL schema, the equivalent rules need to be added as asserts/discriminators. |
Thank you mqjeff and timber, I have used the DFDL and discriminator, working perfectly. Thanks again! |
|
| Back to top |
|
|
|
|
