ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexWebSphere Message Broker SupportIssue modeling DIME format in DFDL

Post new topicReply to topic
Issue modeling DIME format in DFDL View previous topic :: View next topic
Author Message
Be2Be
PostPosted: Sun Feb 11, 2018 4:55 am Post subject: Issue modeling DIME format in DFDL Reply with quote

Newbie

Joined: 11 Feb 2018
Posts: 2

The DIME draft spec (a developerworks link to this doc: cn/webservices/ws-dime/ws-dime.pdf) relates to my question. There is an IETF link as well somewhere.

I'm able to parse an input DIME message with my DFDL model till as far as the DATA_LENGTH field. However, when it comes to parsing the TYPE field (the SOAP v1.1 absolute URI of 41 bytes length), i can't get the entire field value in the TYPE field as DFDL parser considers the leading three null bytes of padding as included in the length. So i get the TYPE field missing the end three bytes 'pe/'. These are then parsed to the next field, DATA. This behavior is not affected by changing the value of property 'Truncate string of specified length' of field TYPE from 'yes' to 'no'.

It is evident that DFDL is not concerned with considering an additional x bytes in the length of a padded field it has truncated of x bytes. This seems intentional as DFDL does not assume knowledge of the quirks of any spec.

DIME spec states that it considers any length value given in the length fields (like TYPE_LENGTH) to exclude any padding (padding with zero octets; for a max of 3 zero octets to achieve alignment to a field length which is a multiple of 4 octets) of the related field.

In the sample message i have been given by the legacy app team, i can see that some fields like ID are not padded to conform to a length of multiple of 4 octets. So the axis library that they are using may be a bit generous in parsing such fields. It looks like i have to do something similar:

TYPE field length property: IF exists (leading three|two|one NUL bytes in current field) then TYPE_LENGTH + 3|2|1 else TYPE_LENGTH

i'm not thinking of using another expression that takes a mod of TYPE_LENGTH/4 and then adds 3 if MOD=1 or adds 2 if MOD=2 etc as i suspect the data i'll get will most likely not conform to the padding rules a la the sample message.

Can the perts please confirm if there is a better way to achieve my goal (i'm assuming my expression above is doable in DFDL)?

My apologies for not posting my sample data and DFDL model snippets as i don't have access to my workstation right now.
Back to top
View user's profile Send private message
timber
PostPosted: Mon Feb 12, 2018 1:56 am Post subject: Reply with quote

Shaman

Joined: 25 Aug 2015
Posts: 729

Good problem description.

It would certainly help if you posted some examples. Probably best to replace nulls with *NULL* in order to make them visible. And please do use [c o d e] tags when posting example messages and/or schema snippets.

My guess is that DFDL can probably handle this, but not using the standard lengthKind=prefixed. You will probably need to parse the length field as a separate field, and then set the length property to a DFDL expression using {some xpath that returns an integer length}.

If the sender is not strictly following the specification then you may have further problems...or it may be easy to be lenient. Hard to say without seeing examples.
Back to top
View user's profile Send private message
Be2Be
PostPosted: Mon Feb 12, 2018 5:57 pm Post subject: Reply with quote

Newbie

Joined: 11 Feb 2018
Posts: 2

timber, I had modelled it as you have suggested (based on IBMdefined/GeneralPurposeFormat.xsd).

TYPE_LENGTH:
Code:
<xsd:element dfdl:alignment="2" dfdl:length="2" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:representation="binary" name="TYPE_LENGTH" type="xsd:unsignedShort"/>


TYPE:
Code:
<xsd:element dfdl:alignment="1" dfdl:length="{xs:int( /DIMEmessage/DimeRec/DIMEHdr/HdrRow1/Type_Length)}" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:textPadKind="padChar" dfdl:textStringJustification="right" dfdl:textStringPadCharacter="%#r00;" dfdl:textTrimKind="padChar" dfdl:truncateSpecifiedLengthString="yes" name="Type" type="xsd:string"/>


Parser Output (truncated just long enough to illustrate):
Code:
<?xml version="1.0" encoding="UTF-8" ?>
  <DIMEmessage xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <DimeRec>
  <DIMEHdr>
  <HdrRow1>
  <version xsi:type="xs:unsignedByte">1</version>
  <MB xsi:type="xs:boolean">true</MB>
  <ME xsi:type="xs:boolean">false</ME>
  <CF xsi:type="xs:boolean">false</CF>
  <Type_T xsi:type="xs:unsignedByte">2</Type_T>
  <Resrvd xsi:type="xs:unsignedByte">0</Resrvd>
  <Opt_Length xsi:type="xs:unsignedShort">0</Opt_Length>
  <ID_Length xsi:type="xs:unsignedShort">41</ID_Length>
  <Type_Length xsi:type="xs:unsignedShort">41</Type_Length>
  </HdrRow1>
  <DataLength xsi:type="xs:unsignedInt">1884</DataLength>
  <ID xsi:type="xs:string">uuid:714C6C40-4531-442E-A498-3AC614200295</ID>
  <Type xsi:type="xs:string">http://schemas.xmlsoap.org/soap/envelo</Type>
  </DIMEHdr>
  <payload xsi:type="xs:string">pe/<![CDATA[*NULL**NULL**NULL*]]><?xml versio


where
Code:
*NULL* = & # 0 ;


Input that gave Output above (Hex):
Code:

         0C 20 00 00 00 29 00 29 00 00 07 5C 75 75 69 64
         3A 37 31 34 43 36 43 34 30 2D 34 35 33 31 2D 34
         34 32 45 2D 41 34 39 38 2D 33 41 43 36 31 34 32
         30 30 32 39 35 00 00 00 68 74 74 70 3A 2F 2F 73
         63 68 65 6D 61 73 2E 78 6D 6C 73 6F 61 70 2E 6F
         72 67 2F 73 6F 61 70 2F 65 6E 76 65 6C 6F 70 65
         2F 00 00 00 3C 3F 78 6D 6C 20 76 65 72 73 69 6F


DIME format (from http://xml.coverpages.org/draft-nielsen-dime-02.txt):
Code:

  0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |         |M|M|C|       |       |                               |
     | VERSION |B|E|F| TYPE_T| RESRVD|         OPTIONS_LENGTH        |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |            ID_LENGTH          |           TYPE_LENGTH         |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                          DATA_LENGTH                          |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               /
     /                     OPTIONS + PADDING                         /
     /                                                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               /
     /                          ID + PADDING                         /
     /                                                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               /
     /                        TYPE + PADDING                         /
     /                                                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               /
     /                        DATA + PADDING                         /
     /                                                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


        Figure 3: DIME Record Layout. The use of "/" indicates a
        field length which is a multiple of 4 octets.


DFDL clearly considers the length units to be the absolute length, not relative to any padding and regardless of whether of whether pad char is requested to be trimmed.

So my question is: Is the only way to make my DFDL model consider that the length units in bytes represent the field length minus any padding is to write some xpath expression that calculates the length units to include the padding byte count for the dfdl:length property of field TYPE (and for other similarly padded fields)?
Back to top
View user's profile Send private message
timber
PostPosted: Tue Feb 13, 2018 4:29 am Post subject: Reply with quote

Shaman

Joined: 25 Aug 2015
Posts: 729

Quote:
DFDL clearly considers the length units to be the absolute length, not relative to any padding and regardless of whether of whether pad char is requested to be trimmed.
That is true always, and regardless of how the length is determined. The length is the length of the 'content region' which always includes padding characters.

However...the null bytes in your message format are not padding. They are alignment bytes. DFDL has specific facilities for controlling alignment and if you set the alignment properties correctly on the OPTIONS, ID, TYPE and DATA elements then it should 'just work'.

As always, the DFDL trace view is invaluable in diagnosing what's going on inside the parser. I recommend using it, if you're not already doing so.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Page 1 of 1

MQSeries.net Forum IndexWebSphere Message Broker SupportIssue modeling DIME format in DFDL
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.