ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexWebSphere Message Broker SupportDFDL Discriminator to Select choice elements for each Occure

Post new topicReply to topic
DFDL Discriminator to Select choice elements for each Occure View previous topic :: View next topic
Author Message
muthu121521
PostPosted: Fri Jan 10, 2020 6:42 am Post subject: DFDL Discriminator to Select choice elements for each Occure Reply with quote

Apprentice

Joined: 31 Aug 2015
Posts: 33

Hi Everyone,
My DFDL looks like below structure

Code:
TABLE ENTRY  1..50
   TABLETYPE String
   TABLETYPEDESC String
   TABLEDATEOPEN String
   CHOICE
     TABLE1INFORMATION1
        FIELD1
        FIELD2
        FIELD2
     TABLE2INFORMATION
        FIELD1
        FIELD2
      TABLE3INFORMATION
        FIELD1
      TABLE4INFORMATION
 

In My above DFDL i need to select the Choice based on the value of the TABLETYPE , but the issue i am facing is when using discriminator at the choice level, I am always getting only first occurrence of the TABLE ENTRY (Root level) Table type value, I am not sure how to get TABLETYPE Value of the current item within the array of TABLE ENTRY
Back to top
View user's profile Send private message
timber
PostPosted: Sat Jan 11, 2020 1:16 pm Post subject: Reply with quote

Grand Master

Joined: 25 Aug 2015
Posts: 1127

I can definitely help with your question, but I don't have enough information yet. Please can you do the following:
- Edit your question and put the structure of the message into [ code ] tags. This will preserve the indentation and allow me to see the the logical structure of your message
- Supply the relevant part of your DFDL xsd. Again, please be sure to use [ code ] tags else it will be unreadable.
Back to top
View user's profile Send private message
muthu121521
PostPosted: Sun Jan 12, 2020 9:01 pm Post subject: Reply with quote

Apprentice

Joined: 31 Aug 2015
Posts: 33

DFDL looks like below structure
Code:

TABLE ENTRY 1..50
   TABLETYPE String
   TABLETYPEDESC String
   TABLEDATEOPEN String
   CHOICE
    TABLE1INFORMATION1
      FIELD1
      FIELD2
      FIELD2
    TABLE2INFORMATION
      FIELD1
      FIELD2
    TABLE3INFORMATION
      FIELD1
    TABLE4INFORMATION


DFDL XSD are as follows
Code:
 
<xsd:element dfdl:lengthKind="implicit" dfdl:occursCountKind="implicit" maxOccurs="50" name="TABLE_ENTRY">
    <xsd:complexType>
        <xsd:sequence>
            <xsd:element default=" " dfdl:length="3" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="TABLETYPE" nillable="true">
                <xsd:annotation>
                    <xsd:appinfo source="http://www.wsadie.com/appinfo">
                        <initialValue kind="SPACE"/>
                    </xsd:appinfo>
                    <xsd:documentation>PIC X(3) display</xsd:documentation>
                </xsd:annotation>
                <xsd:simpleType>
                    <xsd:restriction base="dfdlCobolFmt:PICX__string">
                        <xsd:maxLength value="3"/>
                    </xsd:restriction>
                </xsd:simpleType>
            </xsd:element>
            <xsd:element default=" " dfdl:length="20" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="TABLETYPEDESC" nillable="true">
                <xsd:annotation>
                    <xsd:appinfo source="http://www.wsadie.com/appinfo">
                        <initialValue kind="SPACE"/>
                    </xsd:appinfo>
                    <xsd:documentation>PIC X(20) display</xsd:documentation>
                </xsd:annotation>
                <xsd:simpleType>
                    <xsd:restriction base="dfdlCobolFmt:PICX__string">
                        <xsd:maxLength value="20"/>
                    </xsd:restriction>
                </xsd:simpleType>
            </xsd:element>
            <xsd:element default="0" dfdl:decimalSigned="no" dfdl:length="8" dfdl:nilKind="literalCharacter" dfdl:nilValue="%#r00; " dfdl:textNumberPattern="00000000+" name="TABLEDATEOPEN" nillable="true">
                <xsd:annotation>
                    <xsd:documentation>PIC 9(8) display</xsd:documentation>
                </xsd:annotation>
                <xsd:simpleType>
                    <xsd:restriction base="dfdlCobolFmt:PIC9-Display-Zoned__int">
                        <xsd:minInclusive value="0"/>
                        <xsd:maxInclusive value="99999999"/>
                    </xsd:restriction>
                </xsd:simpleType>
            </xsd:element>
            <xsd:choice dfdl:choiceLength="387" dfdl:choiceLengthKind="explicit">
                <xsd:element dfdl:lengthKind="implicit" name="TABLE1INFORMATION">
                    <xsd:complexType>
                        <xsd:sequence>
                            <xsd:element default="                " dfdl:length="16" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="FIELD1" nillable="true">
                                <xsd:annotation>
                                    <xsd:documentation>PIC X(16) display</xsd:documentation>
                                </xsd:annotation>
                                <xsd:simpleType>
                                    <xsd:restriction base="dfdlCobolFmt:PICX__string">
                                        <xsd:maxLength value="16"/>
                                    </xsd:restriction>
                                </xsd:simpleType>
                            </xsd:element>
                            <xsd:element default="               " dfdl:length="15" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="FIELD2" nillable="true">
                                <xsd:annotation>
                                    <xsd:documentation>PIC X(15) display</xsd:documentation>
                                </xsd:annotation>
                                <xsd:simpleType>
                                    <xsd:restriction base="dfdlCobolFmt:PICX__string">
                                        <xsd:maxLength value="15"/>
                                    </xsd:restriction>
                                </xsd:simpleType>
                            </xsd:element>
                            <xsd:element default="               " dfdl:length="15" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="FIELD3" nillable="true">
                                <xsd:annotation>
                                    <xsd:documentation>PIC X(15) display</xsd:documentation>
                                </xsd:annotation>
                                <xsd:simpleType>
                                    <xsd:restriction base="dfdlCobolFmt:PICX__string">
                                        <xsd:maxLength value="15"/>
                                    </xsd:restriction>
                                </xsd:simpleType>
                            </xsd:element>
                            <xsd:element default="               " dfdl:length="15" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="FIELD4" nillable="true">
                                <xsd:annotation>
                                    <xsd:documentation>PIC X(15) display</xsd:documentation>
                                </xsd:annotation>
                                <xsd:simpleType>
                                    <xsd:restriction base="dfdlCobolFmt:PICX__string">
                                        <xsd:maxLength value="15"/>
                                    </xsd:restriction>
                                </xsd:simpleType>
                            </xsd:element>
                <xsd:element dfdl:lengthKind="implicit" name="TABLE2INFORMATION">
                    <xsd:complexType>
                        <xsd:sequence>
                            <xsd:element default="     " dfdl:length="5" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="FIELD1" nillable="true">
                                <xsd:annotation>
                                    <xsd:documentation>PIC X(5) display</xsd:documentation>
                                </xsd:annotation>
                                <xsd:simpleType>
                                    <xsd:restriction base="dfdlCobolFmt:PICX__string">
                                        <xsd:maxLength value="5"/>
                                    </xsd:restriction>
                                </xsd:simpleType>
                            </xsd:element>
                            <xsd:element default="0" dfdl:decimalSigned="no" dfdl:length="8" dfdl:nilKind="literalCharacter" dfdl:nilValue="%#r00; " dfdl:textNumberPattern="00000000+" name="FIELD2" nillable="true">
                                <xsd:annotation>
                                    <xsd:documentation>PIC 9(8) display</xsd:documentation>
                                </xsd:annotation>
                                <xsd:simpleType>
                                    <xsd:restriction base="dfdlCobolFmt:PIC9-Display-Zoned__int">
                                        <xsd:minInclusive value="0"/>
                                        <xsd:maxInclusive value="99999999"/>
                                    </xsd:restriction>
                                </xsd:simpleType>
                            </xsd:element>                           
                        </xsd:sequence>
                    </xsd:complexType>
                </xsd:element>
                <xsd:element dfdl:lengthKind="implicit" name="TABLE3INFORMATION">
                    <xsd:complexType>
                        <xsd:sequence>
                            <xsd:element default="     " dfdl:length="5" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="FIELD1" nillable="true">
                                <xsd:annotation>
                                    <xsd:documentation>PIC X(5) display</xsd:documentation>
                                </xsd:annotation>
                                <xsd:simpleType>
                                    <xsd:restriction base="dfdlCobolFmt:PICX__string">
                                        <xsd:maxLength value="5"/>
                                    </xsd:restriction>
                                </xsd:simpleType>
                            </xsd:element>
                            <xsd:element default="               " dfdl:length="15" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="FIELD2" nillable="true">
                                <xsd:annotation>
                                    <xsd:documentation>PIC X(15) display</xsd:documentation>
                                </xsd:annotation>
                                <xsd:simpleType>
                                    <xsd:restriction base="dfdlCobolFmt:PICX__string">
                                        <xsd:maxLength value="15"/>
                                    </xsd:restriction>
                                </xsd:simpleType>
                            </xsd:element>
                <xsd:element default=" " dfdl:length="387" dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " name="TABLE4INFORMATION" nillable="true">
                    <xsd:annotation>
                        <xsd:appinfo source="http://www.wsadie.com/appinfo">
                            <initialValue kind="SPACE"/>
                        </xsd:appinfo>
                        <xsd:documentation>PIC X(387) display</xsd:documentation>
                    </xsd:annotation>
                    <xsd:simpleType>
                        <xsd:restriction base="dfdlCobolFmt:PICX__string">
                            <xsd:maxLength value="387"/>
                        </xsd:restriction>
                    </xsd:simpleType>
                </xsd:element>
            </xsd:choice>
        </xsd:sequence>
    </xsd:complexType>                 


But this is the Structure , i am actually not able to get the values of the Tabletype in each iteration of the array , i was able to get only index of the any only one iteration , i was not able to loop through the array in discriminator
Back to top
View user's profile Send private message
timber
PostPosted: Mon Jan 13, 2020 2:01 am Post subject: Reply with quote

Grand Master

Joined: 25 Aug 2015
Posts: 1127

Thanks - that's much clearer now.

The 'TABLETYPE' field is acting exactly like a tag (or 'initiator' in DFDL language). I suggest that you model this as follows:
Code:

message
    ...
    element name="TABLE_ENTRY" maxOccurs=50
        choice initiatedContent="true"
           sequence initiator="ABC"
             group ref="commonFields"  (TABLEDESC AND TABLEOPEN)
             <other fields in 1st choice branch>
           sequence initiator="DEF"
             group ref="commonFields"  (TABLEDESC AND TABLEOPEN)
             <other fields in first 2nd choice branch>
           sequence initiator="GHI"
             group ref="commonFields"  (TABLEDESC AND TABLEOPEN)
             <other fields in 3rd choice branch>
          ...etc


This will also be quite a lot faster to parse because the parser will extract the TABLETYPE field once and resolve the choice immediately (that's what the initiatedContent="true" does).
Back to top
View user's profile Send private message
muthu121521
PostPosted: Mon Jan 13, 2020 7:45 am Post subject: Reply with quote

Apprentice

Joined: 31 Aug 2015
Posts: 33

Hi Timber,

Thanks for Your Reply , due to security issue i was not allowed to put the actual DFDL schema, but the original DFDL schema (Copybook) is very length so i have shortened it to make only the issue highlight but the actual original DFDL looks like below format , each choice has more than 10 fields

Code:

TABLE ENTRY 1..50
   TABLENO  String
   TABLETYPE String
   TABLETYPEDESC String
   TABLEDATEOPEN String
   ELEMENT1   STRING
   .....
   .....
   ELEMENT24  STRING   
   CHOICE
    TABLE1INFORMATION
      ELEMENT1 STRING
      ELEMENT2 STRING
      ...
      ELEMENT13 STRING
      FILLER
    TABLE2INFORMATION
      ELEMENT1 STRING
      ELEMENT2 STRING
      ...
      ELEMENT8 STRING
      FILLER
    TABLE3INFORMATION
      ELEMENT1 STRING
      ELEMENT2 STRING
      ...
      ELEMENT28 STRING
      FILLER
    TABLE4INFORMATION
      ELEMENT1 STRING
      ELEMENT2 STRING
      ...
      ELEMENT3 STRING
      FILLER
   TABLE5INFORMATION
      ELEMENT1 STRING
      ELEMENT2 STRING
      ...
      ELEMENT15 STRING
      FILLER
   TABLE6INFORMATION
      ELEMENT1 STRING
      ELEMENT2 STRING
      ...
      ELEMENT45 STRING
      FILLER
   TABLE7INFORMATION


Each Choice all are of equal length of 387 bytes. Main issue i have faced when i am using discriminator or choice dispatch key , i was not able to determine the Table Type values in the array , when i am trying to use XPath Expression Builder , when i tried to drag Tabletype field from Tabletype entry , its forcing me to give an index, table type value can be different for each occurence of the TABLE ENTRY array, how can i overcome that , not sure of retrieveing the respective table type value?
Back to top
View user's profile Send private message
timber
PostPosted: Mon Jan 13, 2020 8:31 am Post subject: Reply with quote

Grand Master

Joined: 25 Aug 2015
Posts: 1127

OK - thanks for making that clear. I will adjust my advice and make it clearer. If you follow this approach, you will not need to solve your XPath problems because you will not need to write any discriminator expressions. You will need to adjust the structure of the DFDL schema a little, though.

1. If you are using TABLETYPE as a initiator then you need to remove the TABLETYPE element from the DFDL schema. Otherwise DFDL will consume the 3 bytes of TABLETYPE to use as the initiator, then it will try to parse the same 3 bytes again as the TABLETYPE element (but it will actually be consuming the next field instead). Sorry for not explaining that before.
2. If you are using TABLETYPE as an initiator then you do not need any discriminators. The initiator identifies the choice branch automatically.
3. If you are using TABLETYPE as an initiator then you must define the global group 'commonFields' and add a reference to commonFields into every branch of the choice group.
4. If you are using TABLETYPE as an initiator then you must set the initiator on every branch of the choice. The initiator value is whatever value TABLETYPE would have for that choice branch.

So, let me adjust my suggested structure for you:
Code:


-- This global group contains all of the fields _after_ TABLETYPE and _before_ the CHOICE group.
group name="commonFields"
    element name="TABLEDESC"
    element name="TABLEOPEN"
    ELEMENT1   STRING
    .....
    .....
    ELEMENT24  STRING

element name="TABLE_ENTRY" maxOccurs=50
    choice initiatedContent="true"
        sequence initiator="ABC"
            group ref="commonFields"  (TABLEDESC AND TABLEOPEN)
            <other fields in 1st choice branch>
        sequence initiator="DEF"
            group ref="commonFields"  (TABLEDESC AND TABLEOPEN)
            <other fields in first 2nd choice branch>
        sequence initiator="GHI"
            group ref="commonFields"  (TABLEDESC AND TABLEOPEN)
            <other fields in 3rd choice branch>
        ...etc


If you don't want to use this technique, then you will need to find a way write the discriminator expression. I think the position() function is probably the answer. But I would try using the initiators first - it will use a lot less CPU.
Back to top
View user's profile Send private message
muthu121521
PostPosted: Mon Jan 13, 2020 9:38 am Post subject: Reply with quote

Apprentice

Joined: 31 Aug 2015
Posts: 33

We have always used the copybook and converted into DFDL Schema in IIB Creating new message model utility , Can you give me sample dfdl group reference declaration ?
Can we provide multiple values to check if either ABC or BED Then TABLE1Information ?
Back to top
View user's profile Send private message
timber
PostPosted: Tue Jan 14, 2020 1:05 am Post subject: Reply with quote

Grand Master

Joined: 25 Aug 2015
Posts: 1127

Quote:
Can you give me sample dfdl group reference declaration?

You actually need a global group declaration, and then multiple group references (one in each branch of the choice). A DFDL group/group ref is no different from an XSD group/group ref. Just declare the group and reference it in the usual way:
Code:
<xsd:group name="commonFields">
    <xsd:sequence>
        COPY AND PASTE TABLEDESC, TABLEOPEN ETC HERE
    </xsd:sequence>
</xsd:group>


and the group reference:
Code:
<xsd:group ref="myPrefix:commonFields" />
...and (obviously) change myPrefix to whatever the correct prefix is. You can just copy the common fields, with all of their dfdl attributes and annotations, and paste them into the global group.


Quote:
Can we provide multiple values to check if either ABC or BED
So the first choice branch can have two different values for TABLETYPE? No problem - DFDL allows the initiator to be specified as a space-separated list of alternative values. Just specify the "ABC BED" as the initiator value.
Back to top
View user's profile Send private message
muthu121521
PostPosted: Tue Jan 14, 2020 7:48 am Post subject: Reply with quote

Apprentice

Joined: 31 Aug 2015
Posts: 33

Hi Timber,

Thanks for that suggestion, it really worked !! thanks very much for your help on this.
But i am facing couple issues here
1.Last choice which is TABLE4INFORMATION unlike the other choices , it doesn't have any subset of elements , i want this choice to get populated when every other choices failed, what value i can provide here in the initiator?

2.But even if i provide some random value in the initiator , I am getting the below error
Regardless of the value i set i keep getting below errors, I have even tried the quick fix for

Code:
CTDV1561E : When 'initiatedContent' is 'yes' for the sequence or choice, DFDL property 'emptyValueDelimiterPolicy' must not be 'none' or 'terminator'. Element: #xscd


Code:
CTDV1559E : When 'lengthKind' is 'explicit' or 'implicit', DFDL property 'emptyValueDelimiterPolicy' must not be 'initiator' or 'both'



3. I was also trying Below discriminator expression for one Choice example
Code:
{fn:contains( 'ABC:BDE',/TABLE ENTRY[fn:position()]/TABLETYPE}

To determine the choice based on the Table type values in each Table Entry Occurence , i was not able to evaluate the expression, just want to whether it is fn:position() or dfdl:position , because i have tried them both and both of them are not working at all , Just wanted to get the correct rexpression actually
Back to top
View user's profile Send private message
timber
PostPosted: Tue Jan 14, 2020 2:33 pm Post subject: Reply with quote

Grand Master

Joined: 25 Aug 2015
Posts: 1127

1. Ah. That means that we cannot set initiatedContent="true" because that optimisation relies on the parser knowing all of the initiators in advance.
2. The error message is telling ththe current value of emptyValueDelilmiterPolicy is not compatible with initiatedContent="true". However...we will need to set initiatedContent="false" because of 1. So this error should go away.
3. If you want to return to the original DFDL schema structure, and make the discriminators work then you will need the DFDL function dfdl:occursIndex().
See the DFDL specification https://www.ogf.org/documents/GFD.207.pdf for the details of other functions (section 23)

Since you have almost got the initiators working correctly, you might as well try this first:
- Set initiatedContent="false" on the choice group. This should fix the error in your question 2.
- Do not define any initiator for the final choice branch. In fact, you should probably explicitly set it to the empty string since your parsing strategy relies on it being empty.
- In the final choice branch, add the element declaration for TABLETYPE into the sequence group as the first member (immediately before the commonFieldsgroup ref).
Back to top
View user's profile Send private message
muthu121521
PostPosted: Wed Jan 15, 2020 6:34 am Post subject: Reply with quote

Apprentice

Joined: 31 Aug 2015
Posts: 33

Thanks for that suggestion.
Quote:
1. Ah. That means that we cannot set initiatedContent="true" because that optimisation relies on the parser knowing all of the initiators in advance.
2. The error message is telling ththe current value of emptyValueDelilmiterPolicy is not compatible with initiatedContent="true". However...we will need to set initiatedContent="false" because of 1. So this error should go away.


If we set initiatedContent="false" or initiatedContent="No" at the root choice level wont it impact for rest of the choices option?

Regarding with the Point 3
Quote:
If you want to return to the original DFDL schema structure, and make the discriminators work then you will need the DFDL function dfdl:occursIndex().
See the DFDL specification https://www.ogf.org/documents/GFD.207.pdf for the details of other functions (section 23)

We are using IIB Version 10 , in the DFDL Section I can see that occursIndex feature coming under unsupported feature as in the below link

https://www.ibm.com/support/knowledgecenter/SSMKHH_10.0.0/com.ibm.etools.mft.doc/df00150_.html

Because I have even tried as you suggested in the discriminator section and it throws me error when i try to use the below expression

Code:
{fn:contains( 'ABC:BDE',/TABLE ENTRY[dfdl:occursIndex()]/TABLETYPE}


Any Correct Expression suggestion would be of hugely helpful
Back to top
View user's profile Send private message
timber
PostPosted: Wed Jan 15, 2020 9:44 am Post subject: Reply with quote

Grand Master

Joined: 25 Aug 2015
Posts: 1127

Quote:
We are using IIB Version 10 , in the DFDL Section I can see that occursIndex feature coming under unsupported feature
Oops! Well spotted - I forgot that occursIndex was not supported.

Quote:
If we set initiatedContent="false" or initiatedContent="No" at the root choice level wont it impact for rest of the choices option?
Yes, but that is not as serious as you think. The property 'initiatedContent' switches on an optimisation in the DFDL parser, and makes it assume that every branch of the choice has a fixed-length initiator, [i]and[/] all initiators must have exactly the same length. This allows the parser to read the initiator bytes once, and quickly compare the resulting byte values against each choice branch.

If you set it to 'No' then the parser must read the initiator once for each branch. If the TABLETYPE bytes for a branch do not match the initiator then DFDL will backtrack to the start of the choice before starting to parse the next branch. Less efficient, but it still works. And you need that behavour in your scenario because final branch does not have an initiator.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Page 1 of 1

MQSeries.net Forum IndexWebSphere Message Broker SupportDFDL Discriminator to Select choice elements for each Occure
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.