| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| EXtra 'LF' added to new lines in DFDL output |
« View previous topic :: View next topic » |
| Author |
Message
|
| petervh1 |
 Posted: Wed Apr 24, 2024 4:41 am Post subject: EXtra 'LF' added to new lines in DFDL output |
 |
|
Centurion
Joined: 19 Apr 2010
Posts: 135
|
HI
Environment: ACE 12.0.7 on Linux
I'm trying to map the following input data:
| Code: |
0,001048,20240409,20240409,,,,,,,,,,,,,,,,,,,,,,,,,,,A99B7F544C521EDEBDC61A51B84A6BC0,,,,,
5,001048,0096,000003,0,0000000000,20240409,071858,0,06001224590999,000000003000,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,,,,0a5516a906134b68a94698637c2f1933,,,,,
0,001048,20240409,20240409,,,,,,,,,,,,,,,,,,,,,,,,,,,A99B7F544C521EDEBDC61A51B84A6BC0,,,,,
5,001048,0096,000004,0,0000000000,20240409,072213,0,06001224757088,000000002000,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,,,,4c5f8e09d5a44d2f8a1bf3f931ca81ee,,,,,
9,001048,20240409, 564,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
My XSD looks like this:
| Code: |
<?xml version="1.0" encoding="UTF-8"?><xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:csv="http://www.ibm.com/dfdl/CommaSeparatedFormat" xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/" xmlns:fn="http://www.w3.org/2005/xpath-functions" xmlns:ibmDfdlExtn="http://www.ibm.com/dfdl/extensions" xmlns:ibmSchExtn="http://www.ibm.com/schema/extensions">
<xsd:import namespace="http://www.ibm.com/dfdl/CommaSeparatedFormat" schemaLocation="IBMdefined/CommaSeparatedFormat.xsd"/>
<xsd:annotation>
<xsd:appinfo source="http://www.ogf.org/dfdl/">
<dfdl:format documentFinalTerminatorCanBeMissing="yes" encoding="{$dfdl:encoding}" ref="csv:CommaSeparatedFormat" occursCountKind="implicit" separator="" />
</xsd:appinfo>
</xsd:annotation>
<xsd:element ibmSchExtn:docRoot="true" name="MM_xxx_Transactions">
<xsd:complexType>
<xsd:sequence dfdl:separator="" dfdl:separatorSuppressionPolicy="trailingEmpty" >
<xsd:element dfdl:fillByte="" dfdl:outputNewLine="{$dfdl:outputNewLine}" dfdl:terminator="%CR;%LF;%WSP*;" maxOccurs="unbounded" name="record" >
<xsd:complexType>
<xsd:sequence dfdl:outputNewLine="{$dfdl:outputNewLine}" dfdl:separator=",">
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field1" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field2" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field3" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field4" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field5" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field6" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field7" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field8" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field9" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field10" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field11" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field12" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field13" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field14" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field15" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field16" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field17" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field18" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field19" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field20" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field21" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field22" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field23" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field24" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field25" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field26" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field27" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field28" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field29" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field30" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field31" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field32" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field33" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field34" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" dfdl:terminator="" nillable="true" name="field35" type="xsd:string"/>
<xsd:element dfdl:emptyValueDelimiterPolicy="terminator" dfdl:nilKind="literalValue" dfdl:useNilForDefault="yes" minOccurs="1" dfdl:nilValue="%ES;" dfdl:fillByte="" nillable="true" name="field36" type="xsd:string"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>
|
The Test Parse Model works successfully.
However, when I write the same file structure out using the same DFDL (after deleting some of the records), I get the following :
| Code: |
0,001048,20240409,20240409,,,,,,,,,,,,,,,,,,,,,,,,,,,A99B7F544C521EDEBDC61A51B84A6BC0,,,,,
5,001048,0096,000003,0,0000000000,20240409,071858,0,06001224590999,000000003000,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,,,,0a5516a906134b68a94698637c2f1933,,,,,
5,001048,0096,000004,0,0000000000,20240409,072213,0,06001224757088,000000002000,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,,,,4c5f8e09d5a44d2f8a1bf3f931ca81ee,,,,,
9,001048,20240409, 564,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
|
So, an extra 'LF' character is being inserted before line 2 and each successive line.
The trace shows this:
| Code: |
2024-04-24 14:35:27.101508 11591 UserTrace BIP5841I: 'Offset: 0. Parsing will start from root element 'MM_xxx_Transactions'.'
2024-04-24 14:35:27.101588 11591 UserTrace BIP5841I: 'The default value of '%LF;' was assigned to variable 'outputNewLine' in namespace 'http://www.ogf.org/dfdl/dfdl-1.0/'. '
2024-04-24 14:35:27.101618 11591 UserTrace BIP5841I: 'Offset: 0. Starting to process element 'MM_xxx_Transactions'.'
2024-04-24 14:35:27.101664 11591 UserTrace BIP5841I: 'Offset: 0. Starting to process element 'record'.'
|
Why is the 'default value %LF' being inserted? |
|
| Back to top |
|
 |
| timber |
| Posted: Mon May 20, 2024 12:19 pm Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
See https://daffodil.apache.org/docs/dfdl/
If you search your DFDL schema for 'outputNewline' you will see that its value is being picked up from a DFDL variable. That variable defaults to %LF; as required by the specification.
Is this causing a problem, or were you just curious? |
|
| Back to top |
|
|
| petervh1 |
| Posted: Mon May 20, 2024 9:27 pm Post subject: |
|
|
Centurion
Joined: 19 Apr 2010
Posts: 135
|
I found the error.
In the Properties of my FileOutput node, I changed the Record definition to "Record is Unmodified Data" instead of "Record is Delimited Data". |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
