| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| DFDL delimited empty string element |
« View previous topic :: View next topic » |
| Author |
Message
|
| msnyder |
 Posted: Wed May 16, 2012 4:54 pm Post subject: DFDL delimited empty string element |
 |
|
Apprentice
Joined: 24 May 2002
Posts: 32
Location: Jacksonville, FL
|
Need a little help understanding if this is a bug or just not defined properly.
| Code: |
<?xml version="1.0" encoding="UTF-8"?><xsd:schema xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/" xmlns:ibmDfdlExtn="http://www.ibm.com/dfdl/extensions" xmlns:ibmSchExtn="http://www.ibm.com/schema/extensions" xmlns:recSepFieldsFmt="http://www.ibm.com/dfdl/RecordSeparatedFieldFormat" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:import namespace="http://www.ibm.com/dfdl/RecordSeparatedFieldFormat" schemaLocation="IBMdefined/RecordSeparatedFieldFormat.xsd"/>
<xsd:annotation>
<xsd:appinfo source="http://www.ogf.org/dfdl/">
<dfdl:format byteOrder="{$dfdl:byteOrder}" encoding="UTF-8" escapeSchemeRef="recSepFieldsFmt:RecordEscapeScheme" occursCountKind="fixed" ref="recSepFieldsFmt:RecordSeparatedFieldsFormat"/>
</xsd:appinfo>
</xsd:annotation>
<xsd:simpleType name="elementType">
<xsd:restriction base="xsd:string">
<xsd:minLength value="0"/>
<xsd:maxLength value="10"/>
</xsd:restriction>
</xsd:simpleType>
<xsd:complexType name="recordType">
<xsd:sequence dfdl:separator="%#124;">
<xsd:element name="element1" type="elementType"/>
<xsd:element name="element2" type="elementType"/>
<xsd:element name="element3" type="elementType"/>
<xsd:element name="element4" type="elementType"/>
</xsd:sequence>
</xsd:complexType>
|
| Code: |
| element1|||element4 |
| Code: |
May 16, 2012 8:43:18 PM info: Offset: 0. Starting to process element 'log'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/schemaElement::log), 53]
May 16, 2012 8:43:18 PM info: Offset: 0. Starting to process element 'record'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/schemaElement::log/type::0/model::sequence/schemaElement::record), 56]
May 16, 2012 8:43:18 PM info: Offset: 0. Starting to process element 'element1'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element1), 58]
May 16, 2012 8:43:18 PM info: Offset: 0. Found delimited value: 'element1' for element 'element1'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element1), 76]
May 16, 2012 8:43:18 PM info: Offset: 8. Finished processing element 'element1'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element1), 58]
May 16, 2012 8:43:18 PM info: Offset: 8. Found separator '|' owned by the 'sequence' group contained within element 'record'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence), 103]
May 16, 2012 8:43:18 PM info: Offset: 9. Starting to process element 'element2'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element2), 58]
May 16, 2012 8:43:18 PM info: Offset: 9. Element 'element2' is missing because the content region is empty.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element2), 85]
May 16, 2012 8:43:18 PM info: Offset: 9. Found delimited value: '' for element 'element2'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element2), 68]
May 16, 2012 8:43:18 PM fatal: CTDP3059E: Element 'element2' has minOccurs='1' and no default value but the input document contained only '0' occurrences.
|
Defining minLength="0" should tell the parser that the element can be empty - shouldn't it? |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Wed May 16, 2012 9:34 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Did you define it as nillable?
You could then specify that it is there but with null value... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu May 17, 2012 1:17 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Excellent question, and put very clearly.
This is a known defect in the version of IBM DFDL that ships with WMB v8.0. If you open a PMR then you will get the fix as soon as it is available.
In the mean time, fjb_saper's suggestion is probably your best option. You will need:
- xs:nillable="true"
- dfdl:nilKind="nilLiteralValue"
- dfdl:nilValue="%ES;"
which will cause any zero-length element to be put into the message tree with a value of NULL. |
|
| Back to top |
|
|
| msnyder |
| Posted: Thu May 17, 2012 4:17 am Post subject: |
|
|
Apprentice
Joined: 24 May 2002
Posts: 32
Location: Jacksonville, FL
|
Unfortunately, the combination recommended along with various others I tried earlier result in the same exception. I'll open a PMR to get the fix.
Also, has any date been set for fix pack 1? Thanks. |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu May 17, 2012 6:12 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
I have just tried this, and it worked for me.
Element definition:
| Code: |
<xsd:element name="elem1" type="xsd:string"
nillable="true" dfdl:nilKind="literalValue" dfdl:nilValue="%ES;"/> |
Resulting DFDL trace:
| Code: |
17 May 2012 15:05:19 info: Offset: 0. Starting to process element 'elem1'.
17 May 2012 15:05:19 info: Offset: 0. Found nil literal value for element 'elem1'. The value of the element will be reported as nil.
17 May 2012 15:05:19 info: Offset: 0. Found nil literal value for element 'elem1'. The value of the element will be reported as nil.
17 May 2012 15:05:19 info: Offset: 0. Finished processing element 'elem1'. |
Can you post your DFDL xsd ( the one with the nil settings ) ? |
|
| Back to top |
|
|
| msnyder |
| Posted: Thu May 17, 2012 6:59 am Post subject: |
|
|
Apprentice
Joined: 24 May 2002
Posts: 32
Location: Jacksonville, FL
|
I even created a new workspace which helped get rid of the variable length text element issue I had 2 weeks ago and still am getting the same exact exception.
| Code: |
<?xml version="1.0" encoding="UTF-8"?><xsd:schema xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/" xmlns:ibmDfdlExtn="http://www.ibm.com/dfdl/extensions" xmlns:ibmSchExtn="http://www.ibm.com/schema/extensions" xmlns:recSepFieldsFmt="http://www.ibm.com/dfdl/RecordSeparatedFieldFormat" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:import namespace="http://www.ibm.com/dfdl/RecordSeparatedFieldFormat" schemaLocation="IBMdefined/RecordSeparatedFieldFormat.xsd"/>
<xsd:annotation>
<xsd:appinfo source="http://www.ogf.org/dfdl/">
<dfdl:format byteOrder="{$dfdl:byteOrder}" encoding="UTF-8" escapeSchemeRef="recSepFieldsFmt:RecordEscapeScheme" occursCountKind="fixed" ref="recSepFieldsFmt:RecordSeparatedFieldsFormat"/>
</xsd:appinfo>
</xsd:annotation>
<xsd:simpleType name="elementType" >
<xsd:restriction base="xsd:string">
<xsd:minLength value="0"/>
<xsd:maxLength value="10"/>
</xsd:restriction>
</xsd:simpleType>
<xsd:complexType name="recordType">
<xsd:sequence dfdl:separator="%#124;">
<xsd:element name="element1" type="xsd:elementType" xsd:nillable="true" dfdl:nilKind="literalValue" dfdl:nilValue="%ES;" />
<xsd:element name="element2" type="xsd:elementType" xsd:nillable="true" dfdl:nilKind="literalValue" dfdl:nilValue="%ES;" />
<xsd:element name="element3" type="xsd:elementType" xsd:nillable="true" dfdl:nilKind="literalValue" dfdl:nilValue="%ES;" />
<xsd:element name="element4" type="xsd:elementType" xsd:nillable="true" dfdl:nilKind="literalValue" dfdl:nilValue="%ES;" />
</xsd:sequence>
</xsd:complexType>
<xsd:element dfdl:outputNewLine="%LF;" ibmSchExtn:docRoot="true" name="log">
<xsd:complexType>
<xsd:sequence dfdl:separator="%#124;">
<xsd:element name="record" type="recordType"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema> |
|
|
| Back to top |
|
|
| msnyder |
| Posted: Thu May 17, 2012 10:53 am Post subject: |
|
|
Apprentice
Joined: 24 May 2002
Posts: 32
Location: Jacksonville, FL
|
Besides my obvious type problem in the previous post, I had to add dfdl:useNilForDefault="yes" for it to work
| Code: |
<xsd:complexType name="recordType">
<xsd:sequence dfdl:separator="%#124;">
<xsd:element name="element1" type="elementType" nillable="true" dfdl:useNilForDefault="yes" dfdl:nilKind="literalValue" dfdl:nilValue="%ES;" />
<xsd:element name="element2" type="elementType" nillable="true" dfdl:useNilForDefault="yes" dfdl:nilKind="literalValue" dfdl:nilValue="%ES;" />
<xsd:element name="element3" type="elementType" nillable="true" dfdl:useNilForDefault="yes" dfdl:nilKind="literalValue" dfdl:nilValue="%ES;" />
<xsd:element name="element4" type="elementType" nillable="true" dfdl:useNilForDefault="yes" dfdl:nilKind="literalValue" dfdl:nilValue="%ES;" />
</xsd:sequence>
</xsd:complexType> |
Although minLength="0" should be able to be sufficient for string, it wouldn't work for all data types, e.g. integer which would need to be approached the aforementioned.
Thanks for your help. |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu May 17, 2012 11:46 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Glad you got it working. Thanks for not giving up - at least your solution is available to other readers of this thread now.
Just to prevent any confusion in future, there should be no need to set useNilforDefault. This is just a symptom of the defect. |
|
| Back to top |
|
|
| msnyder |
| Posted: Thu May 17, 2012 12:46 pm Post subject: |
|
|
Apprentice
Joined: 24 May 2002
Posts: 32
Location: Jacksonville, FL
|
I have hit another issue with this using the definitions above. For input like:
| Code: |
| element1|2 |element3|element4 |
The 2nd element is being parsed as '2' instead of '2 ' with parsing continuing with the space after the '2'. I don't have time to provide the rest of the information, but will as soon as I get home. |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu May 17, 2012 2:34 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| To save me asking later...the DFDL trace is often a very useful source of information. It is usually the first place I look when I get unexplained DFDL behaviour. Maybe you could take a look, and if you think there's anything interesting in the trace, post the relevant section. |
|
| Back to top |
|
|
| kimbert |
| Posted: Fri May 18, 2012 2:31 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Good news about the nil scenario : I reproduced your problem, and found an error in your xsd.
Your element declarations have an attribute 'xsd:nillable'. The 'xsd:' prefix should not be included because it prevents the DFDL component from recognising the attribute. When that error is fixed, the nil values get recognised without any need to set useNilForDefault.
Note that this is correct behaviour by the DFDL component - it is following the letter of the XSD specification. I think the XSD validator should have reported a warning for this common mistake - that would have flagged up the problem in the Problems view. I will follow this up internally. |
|
| Back to top |
|
|
| msnyder |
| Posted: Fri May 18, 2012 5:33 am Post subject: |
|
|
Apprentice
Joined: 24 May 2002
Posts: 32
Location: Jacksonville, FL
|
Yes, one of the best parts of the DFDL functionality in WMB is the test parser. Unfortunately, I can't post my actual message model and didn't have enough time to create a simple example yesterday.
| Code: |
| element1|elem ||element4 |
It looks like the parser is treating the 1st space of the 2nd element as a delimiter, but not the rest of the spaces. I looked at the IBM part of the definition and didn't find any references to %SP; except for padding. So, is this a bug or just an oversight on my part?
| Code: |
<?xml version="1.0" encoding="UTF-8"?><xsd:schema xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/" xmlns:ibmDfdlExtn="http://www.ibm.com/dfdl/extensions" xmlns:ibmSchExtn="http://www.ibm.com/schema/extensions" xmlns:recSepFieldsFmt="http://www.ibm.com/dfdl/RecordSeparatedFieldFormat" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:import namespace="http://www.ibm.com/dfdl/RecordSeparatedFieldFormat" schemaLocation="IBMdefined/RecordSeparatedFieldFormat.xsd"/>
<xsd:annotation>
<xsd:appinfo source="http://www.ogf.org/dfdl/">
<dfdl:format byteOrder="{$dfdl:byteOrder}" encoding="UTF-8" escapeSchemeRef="recSepFieldsFmt:RecordEscapeScheme" occursCountKind="fixed" ref="recSepFieldsFmt:RecordSeparatedFieldsFormat"/>
</xsd:appinfo>
</xsd:annotation>
<xsd:simpleType name="elementType" >
<xsd:restriction base="xsd:string">
<xsd:minLength value="0"/>
<xsd:maxLength value="10"/>
</xsd:restriction>
</xsd:simpleType>
<xsd:complexType name="recordType">
<xsd:sequence dfdl:separator="%#124;">
<xsd:element name="element1" type="elementType" xsd:nillable="true" dfdl:useNilForDefault="yes" dfdl:nilKind="literalValue" dfdl:nilValue="%ES;" />
<xsd:element name="element2" type="elementType" xsd:nillable="true" dfdl:useNilForDefault="yes" dfdl:nilKind="literalValue" dfdl:nilValue="%ES;" />
<xsd:element name="element3" type="elementType" xsd:nillable="true" dfdl:useNilForDefault="yes" dfdl:nilKind="literalValue" dfdl:nilValue="%ES;" />
<xsd:element name="element4" type="elementType" xsd:nillable="true" dfdl:useNilForDefault="yes" dfdl:nilKind="literalValue" dfdl:nilValue="%ES;" />
</xsd:sequence>
</xsd:complexType>
<xsd:element dfdl:outputNewLine="%LF;" ibmSchExtn:docRoot="true" name="log">
<xsd:complexType>
<xsd:sequence dfdl:separator="%#124;">
<xsd:element name="record" type="recordType"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema> |
| Code: |
May 18, 2012 9:26:40 AM info: Offset: 0. Starting to process element 'log'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/schemaElement::log), 53]
May 18, 2012 9:26:40 AM info: Offset: 0. Starting to process element 'record'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/schemaElement::log/type::0/model::sequence/schemaElement::record), 56]
May 18, 2012 9:26:40 AM info: Offset: 0. Starting to process element 'element1'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element1), 58]
May 18, 2012 9:26:40 AM info: Offset: 0. The simple content region of element 'element1' does not match the literal nil value.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element1), 104]
May 18, 2012 9:26:40 AM info: Offset: 0. Found delimited value: 'element1' for element 'element1'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element1), 76]
May 18, 2012 9:26:40 AM info: Offset: 8. Finished processing element 'element1'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element1), 58]
May 18, 2012 9:26:40 AM info: Offset: 8. Found separator '|' owned by the 'sequence' group contained within element 'record'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence), 103]
May 18, 2012 9:26:40 AM info: Offset: 9. Starting to process element 'element2'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element2), 58]
May 18, 2012 9:26:40 AM info: Offset: 9. The simple content region of element 'element2' does not match the literal nil value.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element2), 104]
May 18, 2012 9:26:40 AM info: Offset: 9. Found delimited value: 'elem' for element 'element2'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element2), 72]
May 18, 2012 9:26:40 AM info: Offset: 13. Finished processing element 'element2'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element2), 59]
May 18, 2012 9:26:40 AM info: Offset: 13. Found separator '|' owned by the 'sequence' group contained within element 'record'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence), 104]
May 18, 2012 9:26:40 AM info: Offset: 14. Starting to process element 'element3'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element3), 59]
May 18, 2012 9:26:40 AM info: Offset: 17. Found nil literal value for element 'element3'. The value of the element will be reported as nil.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element3), 117]
May 18, 2012 9:26:40 AM info: Offset: 17. Finished processing element 'element3'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element3), 59]
May 18, 2012 9:26:40 AM info: Offset: 17. Found separator '|' owned by the 'sequence' group contained within element 'record'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence), 104]
May 18, 2012 9:26:40 AM info: Offset: 18. Starting to process element 'element4'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element4), 59]
May 18, 2012 9:26:40 AM info: Offset: 18. Found nil literal value for element 'element4'. The value of the element will be reported as nil.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element4), 117]
May 18, 2012 9:26:40 AM info: Offset: 18. Found nil literal value for element 'element4'. The value of the element will be reported as nil.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element4), 117]
May 18, 2012 9:26:40 AM info: Offset: 18. Finished processing element 'element4'.
[dfdl = /MB_PTCLocomotiveLogs_V1/Empty.xsd, scd = #xscd(/type::recordType/model::sequence/schemaElement::element4), 59]
May 18, 2012 9:26:40 AM fatal: CTDP3062E: An unexpected non-postfix separator '|' occurs in a postfix position at offset '18' in 'sequence' group contained within element 'record', xpath: '/log[1]/record[1]'. |
|
|
| Back to top |
|
|
| kimbert |
| Posted: Fri May 18, 2012 6:01 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
I have a theory...
Please can you check the DFDL property 'separatorPolicy'. I think you will find that it is set to 'suppressed'. I don't think that's what you want - it should be set to 'suppressedAtEndLax'.
When you try to change it, you will probably find that it is being inherited from the IBM-supplied xsd 'IBMdefined/RecordSeparatedFieldFormat.xsd'. You can find out by letting the mouse hover of the picture of a tree on the left of the 'separatorPolicy' property. You will probably be given a link to IBMdefined/RecordSeparatedFieldFormat.xsd
When you have confirmed that, you should edit the default format for your own xsd and set separatorPolicy there ( better than editing IBMdefined/RecordSeparatedFieldFormat.xsd ).
You can do that as follows:
- click on the 'Show all sections' button at the top of the editor.
- Scroll down to the 'Data Formats' section and open the twistie
- click on Default Format. The defaults for the current xsd will appear in the DFDL properties view |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri May 18, 2012 6:26 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| kimbert wrote: |
| I have a theory... |
It could be bunnies? |
|
| Back to top |
|
|
| msnyder |
| Posted: Fri May 18, 2012 6:26 am Post subject: |
|
|
Apprentice
Joined: 24 May 2002
Posts: 32
Location: Jacksonville, FL
|
| Yes, it was separatorPolicy="suppressed" and I changed it to separatorPolicy="suppressedAtEndLax" with the same results. |
|
| Back to top |
|
|
|
|
|
  |
Goto page 1, 2 Next |
Page 1 of 2 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
