| Author |
Message
|
| sumit |
 Posted: Fri May 06, 2016 7:25 am Post subject: DFDL - handling signed fields |
 |
|
Partisan
Joined: 19 Jan 2006
Posts: 398
|
Hi All,
Env-
OS - Linux
IIB - 9005
We have an MRM message set in MB7 (on Windows) which parses the incoming fixed length file coming from an AS/400.
The data has some signed decimal values with virtual decimal points.
The MRM has this setup-
| Code: |
Physical type = External Decimal
Length=15 bytes
Signed = yes
Sign Orientation = Trailing Overpunched
Virtual decimal point = 2 |
With this setup, when the input is
The message set parse it as
As a new target has been added to this AS/400 source application, we are building a new flow in IIB (Linux).
We created a DFDL with these properties-
| Code: |
Length Kind = explicit
Length = 15
Decimal signed = yes
Text number representation = zoned (was trying it with 'standard' earlier, didn't work)
Number pattern = +0000000000000V00 (and also tired with 0000000000000V00+.. hit and try)
Sign Style = asciiStandard
|
However, the message set is not able to parse it and is failing with error
| Quote: |
| Text:CHARACTER:CTDP3053E: Zoned text to number conversion error for element '#xscd(/schemaElement::StdCustomer/type::0/model::sequence/schemaElement::CustCurrBal)': 6143L |
I am trying to find/read, how we can handle this. Any suggestions?
My last resort would be to go back and use message set in IIB as well.
Another thing that we wanted to try is handle it in ESQL by reading the input data in 'String' format.
_________________
Regards
Sumit |
|
| Back to top |
|
 |
| timber |
| Posted: Fri May 06, 2016 8:39 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
Thanks - good problem description.
According to the DFDL specification (http://www.ibm.com/support/knowledgecenter/SSMKHH_9.0.0/com.ibm.dfdl.spec.doc/dfdl_index.htm?lang=en) you need the following properties:
- textNumberRepresentation=Zoned ( 'Standard' means no overpunched sign will be expected )
- textZonedSignStyle is irrelevant because your character encoding is not an ASCII-derived encoding. I can tell that because your sign character is an 'L'.
(See table '13.6 Properties Specific to Number with Text Representation' in the DFDL spec.)
- The textNumberPattern property should *end* with a + to indicate that the sign is carried on the last digit. This is documented in section 13.6.2.
If that combination of properties is producing an error or the wrong result then you are justified in opening a PMR.
Do not try to write yourself a number parser in ESQL. That might be some people's idea of fun, but it would be a very poor implementation choice. The MRM parser is not deprecated, so that would be a far better fall-back position.
btw, two (rather picky) points about your terminology:
1. A message set does not parse anything because it is data, not code. The MRM parser does the parsing.
2. If you are using DFDL then you should talk about a DFDL schema or a DFDL model. Definitely not a message set. |
|
| Back to top |
|
|
| sumit |
| Posted: Fri May 06, 2016 10:59 am Post subject: |
|
|
Partisan
Joined: 19 Jan 2006
Posts: 398
|
| timber wrote: |
textNumberRepresentation=Zoned ( 'Standard' means no overpunched sign will be expected )
- textZonedSignStyle is irrelevant because your character encoding is not an ASCII-derived encoding. I can tell that because your sign character is an 'L'.
(See table '13.6 Properties Specific to Number with Text Representation' in the DFDL spec.)
- The textNumberPattern property should *end* with a + to indicate that the sign is carried on the last digit. This is documented in section 13.6.2.
If that combination of properties is producing an error or the wrong result then you are justified in opening a PMR. |
Thanks Timber. I tried this already, it didn't work. Yet tried again, no luck. Time for a PMR.
| timber wrote: |
btw, two (rather picky) points about your terminology:
1. A message set does not parse anything because it is data, not code. The MRM parser does the parsing.
2. If you are using DFDL then you should talk about a DFDL schema or a DFDL model. Definitely not a message set. |
Sure. Thanks again!
I will use MRM as we know it is parsing message for an existing flow.
_________________
Regards
Sumit |
|
| Back to top |
|
|
| sumit |
| Posted: Sat May 07, 2016 4:23 am Post subject: |
|
|
Partisan
Joined: 19 Jan 2006
Posts: 398
|
| sumit wrote: |
| Time for a PMR. |
Well, this looks like working!
I had a copy of the message with signed values (without MQMD) and that kept failing again and again. (I kept the 'payload' and was using rfhutil to test it on my local)
Yesterday, got a full extract of the message (i.e. with MQMD and all other headers). When I ran it today, The DFDL schema is able to parse and handle the signed negative value. I will test some more and will update.
_________________
Regards
Sumit |
|
| Back to top |
|
|
| timber |
| Posted: Sun May 08, 2016 11:25 pm Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| Quote: |
| Yesterday, got a full extract of the message (i.e. with MQMD and all other headers). When I ran it today, The DFDL schema is able to parse and handle the signed negative value. |
The MQMD header will specify the CCSID ( character encoding ) of the data. Sounds as if your header-less RFHUtil test was specifying the wrong encoding. Did you specify 'UTF-8' by any chance? ( that's an ASCII-derived encoding, and would not work for EBCDIC data ). |
|
| Back to top |
|
|
| sumit |
| Posted: Mon May 09, 2016 8:19 am Post subject: |
|
|
Partisan
Joined: 19 Jan 2006
Posts: 398
|
| timber wrote: |
| The MQMD header will specify the CCSID ( character encoding ) of the data. Sounds as if your header-less RFHUtil test was specifying the wrong encoding. Did you specify 'UTF-8' by any chance? ( that's an ASCII-derived encoding, and would not work for EBCDIC data ). |
No, I did not. I was thinking that DFDL parser would only look at the payload. Never thought about the *interpretation* of that payload combined with CCSID.
So, the actual data (one with MQMD) was having CCSID 37 against the one I initially struggled with, had 437.
May I say, this suggests that we cannot *always* use DFDL's 'Test Parse Model' to test the validity of data!
_________________
Regards
Sumit |
|
| Back to top |
|
|
| timber |
| Posted: Mon May 09, 2016 10:11 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| Quote: |
| May I say, this suggests that we cannot *always* use DFDL's 'Test Parse Model' to test the validity of data! |
You may, but I would have to disagree because the Test Parse dialog offers you a field where you can enter the encoding that you want to use. |
|
| Back to top |
|
|
|
|
