| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| Escaping a Character in DFDL |
« View previous topic :: View next topic » |
| Author |
Message
|
| kimbert |
 Posted: Mon Sep 29, 2014 12:08 pm Post subject: |
 |
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| kimbert wrote: |
For future reference, it help to provide all of the information up front. Otherwise we just end up asking for it anyway.
You solution ( putting the final field into its own sequence group with the separator set to the empty string ) was a good idea. Shame you did not mention it before. However, you got the structure wrong. I should look like this:
| Code: |
<xsd:element name="Transactions" maxOccurs="unbounded" minOccurs="0">
<xsd:complexType>
<xsd:sequence dfdl:separator="">
<xsd:element name="TransactionDetailRecord" minOccurs="1"
dfdl:initiator="16," dfdl:lengthKind="implicit" dfdl:terminator="%CR;%LF;" >
<xsd:complexType>
<xsd:sequence>
<xsd:sequence dfdl:separator=",">
<xsd:element maxOccurs="1" ref="ns0:TransactionTypeCodes"/>
<xsd:element minOccurs="0" ref="ns0:BankReferenceNumber"/>
<xsd:element minOccurs="0" ref="ns0:CustomerReferenceNumber"/>
</xsd:sequence>
<xsd:sequence dfdl:separator="">
<xsd:element name="Text" type="xsd:string"/>
</xsd:sequence>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element> |
The important changes are:
- The comma-separated fields are inside a nested sequence group of their own.
- the lengthKind of the complex element is changed to 'implicit' ( just believe me on this one - it's the correct option for almost any complex element in DFDL ).
- I removed some properties that should really be located in the default format block. Mainly to make the schema more readable for this forum. |
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
 |
| TheYodas |
| Posted: Mon Sep 29, 2014 2:48 pm Post subject: |
|
|
Novice
Joined: 28 Apr 2014
Posts: 15
|
Now then, I did as you suggested. It is not expecting "comma" before Text field now. I tested using DFDL Test on Eclipse. Here is the error trace:
| Quote: |
Sep 29, 2014 6:24:54 PM info: Offset: 176. Found delimited value: '0023455' for element 'CustomerReferenceNumber'. The delimiter was ','.
[dfdl = /BAI_MsgSet/BAI/BAI_File.xsd, scd = #xmlns(p="btmu:bai")xscd(/schemaElement::p:CustomerReferenceNumber), 116]
Sep 29, 2014 6:24:54 PM info: Offset: 183. Finished processing element 'CustomerReferenceNumber'.
[dfdl = /BAI_MsgSet/BAI/BAI_File.xsd, scd = #xmlns(p="btmu:bai")xscd(/schemaElement::p:CustomerReferenceNumber), 75]
Sep 29, 2014 6:24:54 PM error: CTDP3062E: An unexpected non-postfix separator ',' occurs in a postfix position at offset '183' in 'sequence' group contained within element 'sequence', xpath: '/BAIFile[1]/Group[1]/Account[1]/Transactions[1]/TransactionDetailRecord[1]'.
Sep 29, 2014 6:24:54 PM info: Offset: 183. Parser was unable to resolve data on the current branch and will evaluate the next available branch beginning at offset '144' owned by the 'sequence' group contained within element 'Account'.
[dfdl = /BAI_MsgSet/BAI/BAI_File.xsd, scd = #xmlns(p="btmu:bai")xscd(/schemaElement::p:BAIFile/type::0/model::sequence/schemaElement::Group/type::0/model::sequence/schemaElement::Account/type::0/model::sequence), 212]
|
Here is the record:
| Quote: |
16,115,1234455,S,0,34344,34343,,0023455,NBR OF ITEMS: 11,PROCESS SITE: SOMETHING
|
After "CustomerReferenceNumber" field it is not expecting a comma. I cannot use comma terminator fpr this sequence group as comma is used as seperator.
Why would escapescheme not work for this case? |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Sep 30, 2014 1:51 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Just put a separator on the enclosing sequence - I forgot to do that.
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
|
| TheYodas |
| Posted: Tue Sep 30, 2014 6:19 am Post subject: |
|
|
Novice
Joined: 28 Apr 2014
Posts: 15
|
I should have mentioned that in my previous post. I did try that too. It didnt work. Here is the DFDL definition:
| Code: |
<xsd:element dfdl:lengthKind="delimited" dfdl:occursCountKind="implicit" maxOccurs="unbounded" minOccurs="0" name="Transactions">
<xsd:complexType>
<xsd:sequence dfdl:separator="">
<xsd:element dfdl:emptyValueDelimiterPolicy="none" dfdl:initiator="16," dfdl:lengthKind="implicit" dfdl:terminator="%CR;%LF;" minOccurs="1" name="TransactionDetailRecord">
<xsd:complexType>
<xsd:sequence dfdl:separator=",">
<xsd:sequence dfdl:separator=",">
<xsd:element dfdl:lengthKind="implicit" maxOccurs="1" ref="ns0:TransactionTypeCodes"/>
<xsd:element dfdl:occursCountKind="implicit" minOccurs="0" ref="ns0:BankReferenceNumber"/>
<xsd:element dfdl:occursCountKind="implicit" minOccurs="0" ref="ns0:CustomerReferenceNumber"/>
</xsd:sequence>
<xsd:sequence dfdl:separator="">
<xsd:element minOccurs="0" name="Text" type="xsd:string"/>
</xsd:sequence>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
|
Here is the error:
| Quote: |
Sep 30, 2014 9:52:12 AM info: Offset: 183. Found separator ',' owned by the 'sequence' group contained within element 'TransactionDetailRecord'.
[dfdl = /BAI_MsgSet/BAI/BAI_File.xsd, scd = #xmlns(p="btmu:bai")xscd(/schemaElement::p:BAIFile/type::0/model::sequence/schemaElement::Group/type::0/model::sequence/schemaElement::Account/type::0/model::sequence/schemaElement::Transactions/type::0/model::sequence/schemaElement::TransactionDetailRecord/type::0/model::sequence), 122]
Sep 30, 2014 9:52:12 AM info: Offset: 184. Optional element 'Text' encountered. The DFDL parser will return to this position if the element is not present in the data stream.
[dfdl = /BAI_MsgSet/BAI/BAI_File.xsd, scd = #xmlns(p="btmu:bai")xscd(/schemaElement::p:BAIFile/type::0/model::sequence/schemaElement::Group/type::0/model::sequence/schemaElement::Account/type::0/model::sequence/schemaElement::Transactions/type::0/model::sequence/schemaElement::TransactionDetailRecord/type::0/model::sequence/model::sequence[2]/schemaElement::Text), 152]
Sep 30, 2014 9:52:12 AM info: Offset: 184. Starting to process element 'Text'.
[dfdl = /BAI_MsgSet/BAI/BAI_File.xsd, scd = #xmlns(p="btmu:bai")xscd(/schemaElement::p:BAIFile/type::0/model::sequence/schemaElement::Group/type::0/model::sequence/schemaElement::Account/type::0/model::sequence/schemaElement::Transactions/type::0/model::sequence/schemaElement::TransactionDetailRecord/type::0/model::sequence/model::sequence[2]/schemaElement::Text), 56]
Sep 30, 2014 9:52:12 AM info: Offset: 184. Found delimited value: 'NBR OF ITEMS: 11' for element 'Text'. The delimiter was ','.
[dfdl = /BAI_MsgSet/BAI/BAI_File.xsd, scd = #xmlns(p="btmu:bai")xscd(/schemaElement::p:BAIFile/type::0/model::sequence/schemaElement::Group/type::0/model::sequence/schemaElement::Account/type::0/model::sequence/schemaElement::Transactions/type::0/model::sequence/schemaElement::TransactionDetailRecord/type::0/model::sequence/model::sequence[2]/schemaElement::Text), 113]
Sep 30, 2014 9:52:12 AM info: Offset: 207. Finished processing element 'Text'.
[dfdl = /BAI_MsgSet/BAI/BAI_File.xsd, scd = #xmlns(p="btmu:bai")xscd(/schemaElement::p:BAIFile/type::0/model::sequence/schemaElement::Group/type::0/model::sequence/schemaElement::Account/type::0/model::sequence/schemaElement::Transactions/type::0/model::sequence/schemaElement::TransactionDetailRecord/type::0/model::sequence/model::sequence[2]/schemaElement::Text), 56]
Sep 30, 2014 9:52:12 AM error: CTDP3062E: An unexpected non-postfix separator ',' occurs in a postfix position at offset '207' in 'sequence' group contained within element 'TransactionDetailRecord', xpath: '/BAIFile[1]/Group[1]/Account[1]/Transactions[1]/TransactionDetailRecord[1]'.
Sep 30, 2014 9:52:12 AM info: Offset: 207. Parser was unable to resolve data on the current branch and will evaluate the next available branch beginning at offset '144' owned by the 'sequence' group contained within element 'Account'.
[dfdl = /BAI_MsgSet/BAI/BAI_File.xsd, scd = #xmlns(p="btmu:bai")xscd(/schemaElement::p:BAIFile/type::0/model::sequence/schemaElement::Group/type::0/model::sequence/schemaElement::Account/type::0/model::sequence), 212]
Sep 30, 2014 9:52:12 AM info: Offset: 144. Occurrence '1' of element 'Transactions' was not found in the data. occursCountKind is 'implicit' so no more occurrences of this element will be expected.
[dfdl = /BAI_MsgSet/BAI/BAI_File.xsd, scd = #xmlns(p="btmu:bai")xscd(/schemaElement::p:BAIFile/type::0/model::sequence/schemaElement::Group/type::0/model::sequence/schemaElement::Account/type::0/model::sequence/schemaElement::Transactions), 176]
|
Record submitted:
| Quote: |
| 16,115,1234455,S,0,34344,34343,,0023455,NBR OF ITEMS: 11,PROCESS SITE: SOMETHING |
It is considering comma within Text field as delimiter now.[/quote] |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Oct 01, 2014 1:05 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Sounds like a bug. I have managed to get it working like this:
outer sequence : no separator
first inner sequence : separator="," and separatorPosition="postfix"
second inner sequence : separator=""
Give that a try, and let us know how it goes.
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
|
| TheYodas |
| Posted: Wed Oct 01, 2014 6:50 am Post subject: |
|
|
Novice
Joined: 28 Apr 2014
Posts: 15
|
 Thank you Kimbert. This worked. 
Looks like a bug as you mentioned.
Question, why doesn't Escape scheme work in this case? My understanding was for my scenario Escapescheme would have helped. Looks like my understanding is wrong. Can you enlighten me on this please. |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Oct 01, 2014 4:03 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
I think you have misunderstood what an 'escape scheme' is.
An escape character ( or characters ) allows a delimiter to appear within the value of a delimited field. Think of the Unix '\'. Or the standard CSV escape scheme: one,"two,2",three
In your data format, the ',' in the final field is not escaped at all. So the escape scheme is not relevant (the rules of your data format say that the final field is only delimited by CR/LF - so there is no need to escape the ',' when it appears in the final field )
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
|
| TheYodas |
| Posted: Thu Oct 02, 2014 6:46 am Post subject: |
|
|
Novice
Joined: 28 Apr 2014
Posts: 15
|
Thanks for the confirmation Kimbert.
In case the field was in the middle of the record then I have to use EscapeScheme. Is this right? |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu Oct 02, 2014 8:17 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
That's correct. The final field is a special case, and your data format is taking advantage of that.
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
