ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Parsing a file - DFDL

Post new topic  Reply to topic
 Parsing a file - DFDL « View previous topic :: View next topic » 
Author Message
mqbrks
PostPosted: Tue Feb 28, 2017 11:29 am    Post subject: Parsing a file - DFDL Reply with quote

Voyager

Joined: 17 Jan 2012
Posts: 75

Hi All,

I have a file which has 1 audit record and multiple data records(category1(min occurs 1),category2(min occurs 1),,category3(unbounded))

How to describe the data in DFDL, so that it intelligently takes multiple category 3 records ?

Sample File Contents below

LOGKOING 80TD 1 PR00D 16200008
0113TD1KRE 1 KIMH 1 1 87888FTM -UKK3
0213TD1KRE 1 KIMH 1 1 87888FTM -UKK3
0313TD1KRE 1 KIMH 1 1 87888FTM -UKK3 %HIKK
0313TD1KRE 1 KIMH 1 1 87888FTM -UKK3 $JKHKH
0113TD1KRE 1 KIMH 1 1 87888FTM -UKK3
0213TD1KRE 1 KIMH 1 1 87888FTM -UKK3
0313TD1KRE 1 KIMH 1 1 87888FTM -UKK3 %HIKK
0313TD1KRE 1 KIMH 1 1 87888FTM -UKK3 $JKHKH

LOG - Audit record
011 - category 1
021 - Category 2
031
032 - Category 3 Records

How do I logically define or tell DFDL to parse intelligently based on data to consider the category records?
Back to top
View user's profile Send private message
timber
PostPosted: Tue Feb 28, 2017 1:02 pm    Post subject: Reply with quote

Grand Master

Joined: 25 Aug 2015
Posts: 1280

Good problem description. You did not tell us what you have tried, and what happened. Although I'm guessing that the DFDL parser is attempting to parse everything after Category 2 as a Category3 because it doesn't know when to stop.

The answer is to use a discriminator or an assert to guide the DFDL parser down the right path. You specify an XPath expression which gets evaluated against the parsed data. If it evaluates to false then the parser backs out of that branch ( i.e. Category 3 ) and tries the next possibility ( probably the Category 1 of the next record ).

If you use a discriminator, you get an extra feature. If the XPath evaluates to true then the parser will assume that the current branch is the correct one and will not back out of it. Any parsing errors after that ( within the scope of the same point of uncertainty ) will be treated as an error.
Back to top
View user's profile Send private message
mqbrks
PostPosted: Tue Feb 28, 2017 1:29 pm    Post subject: Reply with quote

Voyager

Joined: 17 Jan 2012
Posts: 75

timber wrote:
Good problem description. You did not tell us what you have tried, and what happened. Although I'm guessing that the DFDL parser is attempting to parse everything after Category 2 as a Category3 because it doesn't know when to stop.


That's exactly right.

Quote:

The answer is to use a discriminator or an assert to guide the DFDL parser down the right path. You specify an XPath expression which gets evaluated against the parsed data. If it evaluates to false then the parser backs out of that branch ( i.e. Category 3 ) and tries the next possibility ( probably the Category 1 of the next record ).



Just now even I got to know that I have to use discriminator or assert to parse the records. Trying to figure out how to use discriminators, do you have any samples ?


Last edited by mqbrks on Tue Feb 28, 2017 1:34 pm; edited 1 time in total
Back to top
View user's profile Send private message
mqjeff
PostPosted: Tue Feb 28, 2017 1:31 pm    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

mqbrks wrote:
timber wrote:
Good problem description. You did not tell us what you have tried, and what happened. Although I'm guessing that the DFDL parser is attempting to parse everything after Category 2 as a Category3 because it doesn't know when to stop.


That's exactly right.

He usually is.

mqbrks wrote:
Just now even I got to know that I have to use discriminator or assert to parse the records. Trying to figure out how to discriminators, do you have any samples ?


mqbrks wrote:
LOG - Audit record
011 - category 1
021 - Category 2
031
032 - Category 3 Records


Among other things, your message seems to be based on a COBOL copybook.

Perhaps you should use that to create your DFDL model...
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
timber
PostPosted: Wed Mar 01, 2017 2:44 am    Post subject: Reply with quote

Grand Master

Joined: 25 Aug 2015
Posts: 1280

Quote:
your message seems to be based on a COBOL copybook.
Perhaps you should use that to create your DFDL model...
That's a good suggestion. It's worth pointing out, though, that the COBOL importer will not automatically create the asserts/discriminators that are required to guide the parser.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Wed Mar 01, 2017 5:04 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

timber wrote:
Quote:
your message seems to be based on a COBOL copybook.
Perhaps you should use that to create your DFDL model...
That's a good suggestion. It's worth pointing out, though, that the COBOL importer will not automatically create the asserts/discriminators that are required to guide the parser.


Well, ok. It'd be nice if it did, but I suppose I have to harangue someone else about that these days.

It does at least build the structures that tell you where to put discriminators/assertions. You might have to add a choice structure though.
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
timber
PostPosted: Wed Mar 01, 2017 5:30 am    Post subject: Reply with quote

Grand Master

Joined: 25 Aug 2015
Posts: 1280

In general it cannot - a COBOL copybook does not contain the information that the parser requires. If writing a COBOL program to process it, that's program logic added by somebody who understands the data. If designing a DFDL schema, the equivalent rules need to be added as asserts/discriminators.
Back to top
View user's profile Send private message
mqbrks
PostPosted: Wed Mar 01, 2017 1:06 pm    Post subject: Reply with quote

Voyager

Joined: 17 Jan 2012
Posts: 75

timber wrote:
In general it cannot - a COBOL copybook does not contain the information that the parser requires. If writing a COBOL program to process it, that's program logic added by somebody who understands the data. If designing a DFDL schema, the equivalent rules need to be added as asserts/discriminators.


Thank you mqjeff and timber, I have used the DFDL and discriminator, working perfectly. Thanks again!
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Parsing a file - DFDL
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.