| Author |
Message
|
| sridharratna |
 Posted: Wed Mar 18, 2015 10:43 pm Post subject: DFDL handle blank rows at end of file |
 |
|
Newbie
Joined: 18 Mar 2015
Posts: 4
|
Hi,
I am modeling a pipe delimited text using DFDL. I am able to handle blank records in the middle of the file, but throwing error when there are blank lines at the end of file.
CTDP3108E: When parsing, the IBM DFDL processor does not support an empty or missing element with a default value.
ParsedDataRegion[SimpleContent, startOffset = 336, length = 13, scd = #xscd(/schemaElement::Employee/type::0/model::sequence/schemaElement::body/type::0/model::sequence/schemaElement::telephone)]
and in DFTL Test trace
Mar 19, 2015, 12:10:36 PM fatal: CTDP3108E: When parsing, the IBM DFDL processor does not support an empty or missing element with a default value.
| Code: |
<?xml version="1.0" encoding="UTF-8"?><xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/" xmlns:ibmDfdlExtn="http://www.ibm.com/dfdl/extensions" xmlns:ibmSchExtn="http://www.ibm.com/schema/extensions" xmlns:recSepFieldsFmt="http://www.ibm.com/dfdl/RecordSeparatedFieldFormat">
<xsd:import namespace="http://www.ibm.com/dfdl/RecordSeparatedFieldFormat" schemaLocation="IBMdefined/RecordSeparatedFieldFormat.xsd"/>
<xsd:annotation>
<xsd:appinfo source="http://www.ogf.org/dfdl/">
<dfdl:format documentFinalTerminatorCanBeMissing="yes" encoding="{$dfdl:encoding}" escapeSchemeRef="recSepFieldsFmt:RecordEscapeScheme" occursCountKind="implicit" ref="recSepFieldsFmt:RecordSeparatedFieldsFormat" useNilForDefault="yes"/>
</xsd:appinfo>
</xsd:annotation>
<xsd:element ibmSchExtn:docRoot="true" name="Employee">
<xsd:complexType>
<xsd:sequence dfdl:separator="%CR;%LF;%WSP*;" dfdl:separatorSuppressionPolicy="anyEmpty" dfdl:terminator="">
<xsd:element name="header">
<xsd:complexType>
<xsd:sequence dfdl:separator="%#124;">
<xsd:element default="id" ibmDfdlExtn:sampleValue="id" name="id" type="xsd:string"/>
<xsd:element default="name" ibmDfdlExtn:sampleValue="name" name="name" type="xsd:string"/>
<xsd:element default="email" ibmDfdlExtn:sampleValue="email" name="email" type="xsd:string"/>
<xsd:element default="doj" ibmDfdlExtn:sampleValue="doj" name="doj" type="xsd:string"/>
<xsd:element dfdl:initiator="" ibmDfdlExtn:sampleValue="address" name="address">
<xsd:complexType>
<xsd:sequence dfdl:initiator="(" dfdl:terminator=")">
<xsd:element name="line1" type="xsd:string"/>
<xsd:element name="line2" type="xsd:string"/>
<xsd:element name="city" type="xsd:string"/>
<xsd:element name="zip" type="xsd:string"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element default="dept" ibmDfdlExtn:sampleValue="dept" name="dept" type="xsd:string"/>
<xsd:element name="telephone" type="xsd:string"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element maxOccurs="unbounded" name="body">
<xsd:complexType>
<xsd:sequence dfdl:separator="%#124;">
<xsd:element default="" ibmDfdlExtn:sampleValue="body_value1" name="id" type="xsd:string"/>
<xsd:element default="" ibmDfdlExtn:sampleValue="body_value2" name="name" type="xsd:string"/>
<xsd:element default="" ibmDfdlExtn:sampleValue="body_value3" name="email" type="xsd:string"/>
<xsd:element default="" ibmDfdlExtn:sampleValue="body_value4" name="doj" type="xsd:string"/>
<xsd:element dfdl:initiator="" dfdl:terminator="" ibmDfdlExtn:sampleValue="body_value5" name="address">
<xsd:complexType>
<xsd:sequence dfdl:initiator="(" dfdl:terminator=")">
<xsd:element default="" ibmDfdlExtn:sampleValue="" name="line1" type="xsd:string"/>
<xsd:element default="" ibmDfdlExtn:sampleValue="" name="line2" type="xsd:string"/>
<xsd:element default="" ibmDfdlExtn:sampleValue="" name="city" type="xsd:string"/>
<xsd:element default="" ibmDfdlExtn:sampleValue="" name="zip" type="xsd:string"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element default="" ibmDfdlExtn:sampleValue="body_value6" name="dept" type="xsd:string"/>
<xsd:element name="telephone" type="xsd:string"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>
|
|
|
| Back to top |
|
 |
| kimbert |
| Posted: Thu Mar 19, 2015 1:44 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Have you set documentFinalTerminatorCanBeMissing="yes" in the format block?
Have you taken a DFDL Trace?
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
|
| sridharratna |
| Posted: Thu Mar 19, 2015 3:52 am Post subject: |
|
|
Newbie
Joined: 18 Mar 2015
Posts: 4
|
| documentFinalTerminatorCanBeMissing will handle only blank lines after header or in the middle of the data. But is not handling blank lines at the end of file. |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu Mar 19, 2015 4:52 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
That response is illogical. Please
- read what the DFDL specification says about documentFinalTerminatorCanBeMissing
- explain your last response in terms of the DFDL specification
Alternatively, provide some detailed information on how and where the failure happens. That will probably mean that you need to read the DFDL Trace and quote the relevant part(s) in your next post.
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
|
| sridharratna |
| Posted: Thu Mar 19, 2015 10:25 pm Post subject: |
|
|
Newbie
Joined: 18 Mar 2015
Posts: 4
|
as per your guidance gone thru DFDl config in details and common errors... and i did the same common error
I was doing a mistake of keeping terminator "%CR;%LF;%WSP*;" sequence element before header instead keeping at header level. |
|
| Back to top |
|
|
| shanson |
| Posted: Fri Mar 20, 2015 6:52 am Post subject: |
|
|
Partisan
Joined: 17 Oct 2003
Posts: 344
Location: IBM Hursley
|
Correct. documentFinalTerminatorCanBeMissing only applies to terminator, not to separator. Switching to terminator will handle what you are seeing.
Alternatively, if you prefer to use a separator, and there is always an extra separator at the end of the last record, then you can set separatorPosition 'postfix'. |
|
| Back to top |
|
|
|
|
