| Author |
Message
|
| agrawalamit166 |
 Posted: Wed Sep 07, 2016 1:28 pm Post subject: DFDL parser- String fields has Space or low-Value |
 |
|
Voyager
Joined: 17 Aug 2009
Posts: 78
Location: NY, US
|
Hi,
IIB service gets fixed length message from mainframe using CICS request (EBCDIC 037 - 785). few string fields are recognized NULL value as Space and other string fields are recognized NULL value as low-value.
If I define, recognized NULL value as spaces for all fields, I am getting character (as square image) for 2-3 string fields.
Also if i manually change recognized NULL value as low-value for those fields, still it shows character as square image.
I am using IIB 9.0.0.4 |
|
| Back to top |
|
 |
| shanson |
| Posted: Wed Sep 07, 2016 11:29 pm Post subject: |
|
|
Partisan
Joined: 17 Oct 2003
Posts: 344
Location: IBM Hursley
|
| It sounds like incorrect DFDL annotations for the 2-3 fields that are not working. Please supply the snippet of DFDL schema that covers those fields. |
|
| Back to top |
|
|
| agrawalamit166 |
| Posted: Thu Sep 08, 2016 9:54 am Post subject: |
|
|
Voyager
Joined: 17 Aug 2009
Posts: 78
Location: NY, US
|
Thanks Shanson,
I found the issue:
I have
| Code: |
| dfdl:nilKind="literalCharacter" dfdl:nilValue="%SP; " |
for String value to recognize null in DFDL xsd.
I was trying to change only nilValue to dfdl:nilValue="%#r00; ", it did not work.
When I change to
| Code: |
| dfdl:nilKind="literalValue" dfdl:nilValue="%#r00; " |
it works fine.
Thanks. |
|
| Back to top |
|
|
| timber |
| Posted: Thu Sep 08, 2016 1:18 pm Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
The DFDL specification says:
| Quote: |
| If literalCharacter then dfdl:nilValue specifies a single character or a single byte that, when repeated to the length of the element, is the nil value. literalCharacter may only be specified for fixed length elements |
In other words, if you want the field to be parsed as NULL in the message tree, it must be entirely filled with lthe nilValue (low-values in your case).
But you have set nilKind="nilLiteralValue". The nilValue is defined to be a single byte (%#r00;). That can only be valid if the field is defined to be exactly 1 byte long. I claim that that your schema should produce a schema definition error unless length="1" and lengthUnits="bytes", because the nilValue is too short to be a valid value for the field. |
|
| Back to top |
|
|
| shanson |
| Posted: Fri Sep 09, 2016 3:58 am Post subject: |
|
|
Partisan
Joined: 17 Oct 2003
Posts: 344
Location: IBM Hursley
|
| Quote: |
| I claim that that your schema should produce a schema definition error unless length="1" and lengthUnits="bytes", because the nilValue is too short to be a valid value for the field. |
No. See the grammar in section 9 of the DFDL spec. Padding character is trimmed before the nil literal value is checked. This is deliberate as it means you can set a nil value in scope via dfdl:format without worrying about the length of the value. |
|
| Back to top |
|
|
| agrawalamit166 |
| Posted: Mon Sep 12, 2016 7:41 am Post subject: |
|
|
Voyager
Joined: 17 Aug 2009
Posts: 78
Location: NY, US
|
| Quote: |
| That can only be valid if the field is defined to be exactly 1 byte long |
That's not true, I have length >1, and it works fine for me. |
|
| Back to top |
|
|
| timber |
| Posted: Mon Sep 12, 2016 1:23 pm Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
See shanson's explanation - you may have got lucky because of a special rule in DFDL:
| Quote: |
| Padding character is trimmed before the nil literal value is checked. This is deliberate as it means you can set a nil value in scope via dfdl:format without worrying about the length of the value |
True, but the same can be achieved more simply using nilKind='nilLiteralCharacter'. I think nilLiteralCharacter is the simplest way to model a field that is filled with N instances of the same character.
So why did nilLiteralCharacter not work? The only reasons I can think of are
a) a defect in the DFDL parser or
b) the field contains a mixture of the nilValue and some other characters (perhaps padding, perhaps not).
If it was b), then it might explain the presence of the unexpected 'square images' in the field values. I still suspect that this thread and the other one are just two aspects of the same issue. |
|
| Back to top |
|
|
| agrawalamit166 |
| Posted: Tue Sep 13, 2016 7:58 am Post subject: |
|
|
Voyager
Joined: 17 Aug 2009
Posts: 78
Location: NY, US
|
| Quote: |
| but the same can be achieved more simply using nilKind='nilLiteralCharacter'. I think nilLiteralCharacter is the simplest way to model a field that is filled with N instances of the same character. |
I am getting following error message:
| Code: |
| CTDM2202E : DFDL property value is not valid. Property 'nilKind', value 'nilLiteralCharacter', object 'xsd:element #FIELD_NAME'. XSD_NAME.xsd /DFDL_PATH line 686 DFDL Validation Problem |
| Quote: |
| the field contains a mixture of the nilValue and some other characters (perhaps padding, perhaps not). |
You are correct, this is what happening in other message thread. I am getting nul character with actual response message. |
|
| Back to top |
|
|
| timber |
| Posted: Tue Sep 13, 2016 8:28 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
Sorry for the confusion - I meant nilKind='literalCharacter'.
I think we now have all the facts.
- You have some fields that, when nil, are entirely filled with low-values. You should model those with nilKind='literalCharacter' and nilValue='%#r00'
- You have some fields that, when nil, are entirely filled with spaces. You should model those with nilKind='literalCharacter' and nilValue='%SP;'
- You have some fields that, when nil, contain one or more low-valued bytes followed by spaces for padding. You should model those with nilKind='literalvalue' nilValue='%#r00' textStringPadCharacter="%SP;"
That last example may be the wrong way round, but hopefully you get the idea. If you need to ask further questions on this thread, please ensure that you include the entire field value (actual hexadecimal byte values) in your next post. |
|
| Back to top |
|
|
| agrawalamit166 |
| Posted: Tue Sep 13, 2016 9:34 am Post subject: |
|
|
Voyager
Joined: 17 Aug 2009
Posts: 78
Location: NY, US
|
Thanks Timber, for summarize things.
i just checked 3rd scenario (Mixed fields, contains message with nul character)
I defined
| Code: |
<dfdl:format binaryFloatRep="{$dfdl:binaryFloatRep}" byteOrder="{$dfdl:byteOrder}" encoding="{$dfdl:encoding}" ignoreCase="yes" leadingSkip="0" occursCountKind="fixed" ref="dfdlCobolFmt:CobolDataFormat" textNumberPadCharacter="0" textStringJustification="left" textStringPadCharacter="%#r00;" textZonedSignStyle="asciiStandard" trailingSkip="0" useNilForDefault="no"/>
|
and at field level
| Code: |
| <xsd:element default=" " dfdl:length="200" dfdl:nilKind="literalValue" dfdl:nilValue="%#r00; " dfdl:textStringPadCharacter="%SP; " name="MESSAGE" nillable="true"> |
But it still shows same error: and I getting message with nul character.
| Code: |
| An invalid XML character (Unicode: 0x0) was found in the element content of the document. |
|
|
| Back to top |
|
|
| timber |
| Posted: Tue Sep 13, 2016 10:55 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
Ahem.
| Quote: |
| If you need to ask further questions on this thread, please ensure that you include the entire field value (actual hexadecimal byte values) in your next post. |
Also, you appear to be testing your DFDL model using the IIB runtime. Please verify that the model is correct in the IIB Toolkit before deploying. It simplifies things greatly (and you can use shanson's tip to see the raw hex values in the message) |
|
| Back to top |
|
|
| agrawalamit166 |
| Posted: Tue Sep 13, 2016 11:52 am Post subject: |
|
|
Voyager
Joined: 17 Aug 2009
Posts: 78
Location: NY, US
|
Thanks a lot...
I saved mainframe response message in file and run against DFDL schema in IIB toolkit. Then I got this response. I was able to parse message successfully.
I verified success vs failure and found that Failure message shows CDATA, contains "&_#_0_;" (Ignore '_' else difficult to show correct data), which is creating an issue : here is details:
Success:
| Code: |
| <MESSAGE xsi:type="xs:string">Data is accepted for process setup. </MESSAGE> |
Failure:
| Code: |
| <MESSAGE xsi:type="xs:string">ERROR7657 - Invalid Action: *REVOKE* Must be SETUP OR AUTH<![CDATA[��������������]]> </MESSAGE> |
|
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Sep 13, 2016 11:58 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Are any of those characters that are not allowed in CDATA sections?
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| agrawalamit166 |
| Posted: Tue Sep 13, 2016 12:10 pm Post subject: |
|
|
Voyager
Joined: 17 Aug 2009
Posts: 78
Location: NY, US
|
| Quote: |
| Are any of those characters that are not allowed in CDATA sections? |
CDATA contains only nul character like I mentioned "&_#0;" (remove underscore)
After CDATA I can see spaces "&_#32;" which are taken care by DFDL schema.
One more thing: when I see data in IIB toolkit (while debugging), it does not show CDATA tag. |
|
| Back to top |
|
|
| timber |
| Posted: Tue Sep 13, 2016 12:18 pm Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| Quote: |
| If you need to ask further questions on this thread, please ensure that you include the entire field value (actual hexadecimal byte values) in your next post. |
I'm serious. I cannot help you unless you supply this information.
You have been told how to obtain this information on the other thread:
| Quote: |
| The DFDL editor test parse/serialize views in the IIB toolkit let you see the hex. On the toolbar for those views, click on the upside-down triangle symbol and select 'Show hex'. |
|
|
| Back to top |
|
|
|
|
