| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| DFDL - Empty fields in pipe-delimited record |
« View previous topic :: View next topic » |
| Author |
Message
|
| petervh1 |
 Posted: Thu Dec 22, 2022 2:41 am Post subject: DFDL - Empty fields in pipe-delimited record |
 |
|
Centurion
Joined: 19 Apr 2010
Posts: 122
|
Hi
I'm setting up a DFDL mapping as follows:
| Quote: |
<?xml version="1.0" encoding="UTF-8"?><xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/" xmlns:fn="http://www.w3.org/2005/xpath-functions" xmlns:ibmSchExtn="http://www.ibm.com/schema/extensions" xmlns:recSepFieldsFmt="http://www.ibm.com/dfdl/RecordSeparatedFieldFormat">
<xsd:import namespace="http://www.ibm.com/dfdl/RecordSeparatedFieldFormat" schemaLocation="IBMdefined/RecordSeparatedFieldFormat.xsd"/>
<xsd:annotation>
<xsd:appinfo source="http://www.ogf.org/dfdl/">
<dfdl:format documentFinalTerminatorCanBeMissing="yes" encoding="{$dfdl:encoding}" escapeSchemeRef="" occursCountKind="implicit" ref="recSepFieldsFmt:RecordSeparatedFieldsFormat"/>
</xsd:appinfo>
</xsd:annotation>
<xsd:element ibmSchExtn:docRoot="true" name="ProductHierarchyCSV">
<xsd:complexType>
<xsd:sequence dfdl:separatorSuppressionPolicy="anyEmpty">
<xsd:element maxOccurs="unbounded" name="body">
<xsd:complexType>
<xsd:sequence dfdl:separator="">
<xsd:element default="body_value1" dfdl:terminator="%#124;" name="body_elem1" type="xsd:string"/>
<xsd:element default="body_value2" dfdl:terminator="%#124;" name="body_elem2" type="xsd:string"/>
<xsd:element dfdl:terminator="%#124;" dfdl:textNumberPattern="0" name="body_elem3" type="xsd:integer"/>
<xsd:element default="body_value4" dfdl:terminator="%#124;" name="body_elem4" type="xsd:string"/>
<xsd:element default="body_value5" dfdl:terminator="%#124;" name="body_elem5" type="xsd:string"/>
<xsd:element default="" dfdl:occursCountKind="implicit" dfdl:terminator="%#124;" minOccurs="0" name="body_elem6" type="xsd:string"/>
<xsd:element default="body_value7" dfdl:terminator="%#124;" name="body_elem7" type="xsd:string"/>
<xsd:element default="true" minOccurs="0" name="body_elem8" type="xsd:boolean"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema> |
When I run a Test Parse using this data:
| Quote: |
Eggs|2|203401|ERP|xyz||/c-438/All-Departments/Drinks|TRUE
|
I get:
| Quote: |
22 Dec 2022, 12:33:39 info: Offset: 0. Parsing will start from root element 'ProductHierarchyCSV'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV), 78]
22 Dec 2022, 12:33:39 info: The default value of '%LF;' was assigned to variable 'outputNewLine' in namespace 'http://www.ogf.org/dfdl/dfdl-1.0/'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = , 133]
22 Dec 2022, 12:33:39 info: Offset: 0. Starting to process element 'ProductHierarchyCSV'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV), 69]
22 Dec 2022, 12:33:39 info: Offset: 0. Up to 'unbounded' occurrences of element 'body' will be expected because occursCountKind='implicit' and maxOccurs='unbounded'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body), 145]
22 Dec 2022, 12:33:39 info: Offset: 0. Starting to process element 'body'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body), 54]
22 Dec 2022, 12:33:39 info: Offset: 0. Starting to process element 'body_elem1'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem1), 60]
22 Dec 2022, 12:33:39 info: Offset: 0. Found delimited value 'Eggs' for element 'body_elem1'. The delimiter was '|'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem1), 97]
22 Dec 2022, 12:33:39 info: Offset: 4. Found terminator '|' for 'body_elem1'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem1), 57]
22 Dec 2022, 12:33:39 info: Offset: 5. Finished processing element 'body_elem1'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem1), 60]
22 Dec 2022, 12:33:39 info: Offset: 5. Starting to process element 'body_elem2'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem2), 60]
22 Dec 2022, 12:33:39 info: Offset: 5. Found delimited value '2' for element 'body_elem2'. The delimiter was '|'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem2), 94]
22 Dec 2022, 12:33:39 info: Offset: 6. Found terminator '|' for 'body_elem2'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem2), 57]
22 Dec 2022, 12:33:39 info: Offset: 7. Finished processing element 'body_elem2'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem2), 60]
22 Dec 2022, 12:33:39 info: Offset: 7. Starting to process element 'body_elem3'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem3), 60]
22 Dec 2022, 12:33:39 info: Offset: 7. Found delimited value '203401' for element 'body_elem3'. The delimiter was '|'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem3), 99]
22 Dec 2022, 12:33:39 info: Offset: 13. The simple content region of element 'body_elem3' was converted to logical value '203401'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem3), 113]
22 Dec 2022, 12:33:39 info: Offset: 13. Found terminator '|' for 'body_elem3'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem3), 58]
22 Dec 2022, 12:33:39 info: Offset: 14. Finished processing element 'body_elem3'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem3), 61]
22 Dec 2022, 12:33:39 info: Offset: 14. Starting to process element 'body_elem4'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem4), 61]
22 Dec 2022, 12:33:39 info: Offset: 14. Found delimited value 'ERP' for element 'body_elem4'. The delimiter was '|'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem4), 97]
22 Dec 2022, 12:33:39 info: Offset: 17. Found terminator '|' for 'body_elem4'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem4), 58]
22 Dec 2022, 12:33:39 info: Offset: 18. Finished processing element 'body_elem4'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem4), 61]
22 Dec 2022, 12:33:39 info: Offset: 18. Starting to process element 'body_elem5'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem5), 61]
22 Dec 2022, 12:33:39 info: Offset: 18. Found delimited value 'Checkers' for element 'body_elem5'. The delimiter was '|'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem5), 102]
22 Dec 2022, 12:33:39 info: Offset: 26. Found terminator '|' for 'body_elem5'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem5), 58]
22 Dec 2022, 12:33:39 info: Offset: 27. Finished processing element 'body_elem5'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem5), 61]
22 Dec 2022, 12:33:39 info: Offset: 27. Optional element 'body_elem6' encountered. The DFDL parser will return to this position if the element is not present in the data stream.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem6), 157]
22 Dec 2022, 12:33:39 info: Offset: 27. Starting to process element 'body_elem6'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem6), 61]
22 Dec 2022, 12:33:39 info: Offset: 27. Element 'body_elem6' is empty because the content region is empty.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem6), 86]
22 Dec 2022, 12:33:39 info: Offset: 27. Found delimited value '' for element 'body_elem6'. The delimiter was '|'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem6), 94]
22 Dec 2022, 12:33:39 info: Offset: 27. Found terminator '|' for 'body_elem6'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem6), 58]
22 Dec 2022, 12:33:39 error: CTDP3133E: An unexpected terminator was found for an empty element 'body_elem6'.
22 Dec 2022, 12:33:39 info: Offset: 28. Parser was unable to resolve data on the current branch and will evaluate the next available branch beginning at offset '27' owned by the 'sequence' group contained within element 'body'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence), 207]
22 Dec 2022, 12:33:39 info: Offset: 27. Occurrence '1' of element 'body_elem6' was not found in the data. occursCountKind is 'implicit' so no more occurrences of this element will be expected.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem6), 173]
22 Dec 2022, 12:33:39 info: Offset: 27. Starting to process element 'body_elem7'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem7), 61]
22 Dec 2022, 12:33:39 info: Offset: 27. Element 'body_elem7' is empty because the content region is empty.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem7), 86]
22 Dec 2022, 12:33:39 info: Offset: 27. Found delimited value '' for element 'body_elem7'. The delimiter was '|'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem7), 94]
22 Dec 2022, 12:33:39 info: Offset: 27. Found terminator '|' for 'body_elem7'.
[dfdl = /HP38A/ProductHierarchyCSV.xsd, scd = #xscd(/schemaElement::ProductHierarchyCSV/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::body_elem7), 58]
22 Dec 2022, 12:33:39 error: CTDP3133E: An unexpected terminator was found for an empty element 'body_elem7'.
22 Dec 2022, 12:33:39 fatal: CTDP3133E: An unexpected terminator was found for an empty element 'body_elem7'.
|
I suspect I'm not understanding something here. From the trace it looks to me that after finding an empty element in body_elem6 the parser is trying to find a delimited value for element body_elem7 and is finding this to be empty.
What do I need to change for the parser to recognise that body_elem6 is empty (quite legal, because it's an optional element), then parse body_elem7 (mandatory) and then parse body_elem8 (which happens to be populated, although it's an optional element) ? |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Thu Dec 22, 2022 8:34 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
optional element does not equate to nillable element. However looking at your input you treat your element 6 as mandatory, nillable and not as optional...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| timber |
| Posted: Tue Jan 03, 2023 1:53 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1280
|
Your data format is CSV, which is a positional format because each field is identified by its position within the record. In a positional format you should set separationSuppressionPolicy to 'trailingEmpty'.
I suggest that you take a look at the actual value for separatorSuppressionPolicy on the inner sequence (by looking in the DFDL editor). |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
