| Author |
Message
|
| andrewfemin |
 Posted: Tue Oct 17, 2017 3:40 am Post subject: DFDL serializer issue in IIB9 |
 |
|
Acolyte
Joined: 26 Aug 2017
Posts: 54
|
Hi,
I've created a DFDL to serialize fixed length data with a Parent element and some child elements. The length of the child elements add up to 100 and some of them have minOccurs=0 and maxOccurs=1. The parent element has minOccurs=1 and maxOccurs=unbounded.
If one of the child element does not have any value, I get less than 100(let's say 90) characters in the output file. As per my requirement, if the data adds up to only 90 characters, 10 spaces should be added and the total length should be 100.
When I tried giving Fill Byte as %SP;, lengthKind as explicit and length as 100 in the parent element, I could get the 10 additional spaces and the total length as 100. But my parent element is repeatable and hence I changed the lengthKind as pattern and the length pattern as (.{100})*. The Fill Byte is still %SP;. But after this change, I'm only getting 90 characters in the output file without the fill bytes.
Thanks a lot for taking time to read this. Please let me know if I'm doing anything wrong here. |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Tue Oct 17, 2017 5:01 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
You need a 3 level structure:
parent element
child element length fixed 100 occurs ...
child structure length implicit
fields in child structure
Hope it helps 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| andrewfemin |
| Posted: Tue Oct 17, 2017 5:26 am Post subject: |
|
|
Acolyte
Joined: 26 Aug 2017
Posts: 54
|
Thanks for the reply.
If I get two sets of values with 90 characters each, having the three level structure will give an output that goes like this:
(90characters)+(10spaces)+(90character)+(10spaces) = 200 in total
But I need it to be;
(90characters)+(90characters)+(20spaces) = 200 in total
I can add a Filler field with length 1 at the end of the dfdl and model my code to have the filler repeat as many times as required. But there must be a way to handle it in the DFDL. I tried with the length pattern. It's not working. Please guide me. Thanks. |
|
| Back to top |
|
|
| timber |
| Posted: Tue Oct 17, 2017 11:37 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| Quote: |
| But my parent element is repeatable and hence I changed the lengthKind as pattern and the length pattern as (.{100})* |
Excellent problem description, but I did not understand that part. Why could you not leave the lengthKind as 'explicit'?
| Quote: |
| If one of the child element does not have any value, I get less than 100(let's say 90) characters in the output file. |
You could fix that by assigning a default value to the child elements. DFDL will write the default value when the element's value does not exist. You would need to ensure that the child elements exist in OutputRoot.DFDL with no value. That is probably quite simple to achieve in your mapping logic. |
|
| Back to top |
|
|
| andrewfemin |
| Posted: Thu Oct 19, 2017 12:14 am Post subject: |
|
|
Acolyte
Joined: 26 Aug 2017
Posts: 54
|
| timber wrote: |
| Excellent problem description, but I did not understand that part. Why could you not leave the lengthKind as 'explicit'? |
If I set the lengthKind as explicit and length as 100 and have the element repeat, my output goes like below:
90 characters + 10 spaces + 90 characters + 10 spaces = 200 in total
But I want it to be like below:
90 characters + 90 characters + 20 spaces = 200 in total
The total could be any multiple of 100. Hence I cannot have it as explicit. So I have set the element's length as pattern and trying to get the Fill Bytes(space) fill up the remaining characters up to the nearest multiple of 100.
| timber wrote: |
| You could fix that by assigning a default value to the child elements. DFDL will write the default value when the element's value does not exist. You would need to ensure that the child elements exist in OutputRoot.DFDL with no value. That is probably quite simple to achieve in your mapping logic. |
Having default values will again give me an output like the one stated above. I do not want the default values to act like placeholders for the missing fields.
Imagine 3 child elements with length 10 each. If there's no value in element 2, having a default value will give the below result:
element1(10 characters)+default value of element 2(10 characters)+element3(10 characters) = 30 in total
But I want the below result:
element1(10 characters)+element3(10 characters)+10 spaces = 30 in total
Please help me out here.
Last edited by andrewfemin on Thu Oct 19, 2017 7:45 pm; edited 1 time in total |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Oct 19, 2017 4:31 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| andrewfemin wrote: |
| timber wrote: |
| Excellent problem description, but I did not understand that part. Why could you not leave the lengthKind as 'explicit'? |
If I set the lengthKind as explicit and length as 100 and have the element repeat, my output goes like below:
90 characters + 10 spaces + 90 characters + 10 spaces = 200 in total
But I want it to be like below:
90 characters + 90 characters + 20 spaces = 200 in total
The total could be any multiple of 100. Hence I cannot have it as explicit. So I have set the element's length as pattern and trying to get the Fill Bytes(space) fill up the remaining characters up to the nearest multiple of 100.
| timber wrote: |
| You could fix that by assigning a default value to the child elements. DFDL will write the default value when the element's value does not exist. You would need to ensure that the child elements exist in OutputRoot.DFDL with no value. That is probably quite simple to achieve in your mapping logic. |
Having default values will again give me an output like the one stated above. I do not want the default values to act like placeholders for the missing fields.
Imagine 3 child elements with length 10 each. If there's no value in element 2, having a default value will give the below result:
element1(10 characters)+default value of element 2(10 characters)+element3(10 characters) = 30 in total
But I want the below result:
element1(10 characters)+element3(10 characters)+10 spaces = 30 in total
Please help me out here.
|
So taking this logic up to 10 records, do you want the output to be - 10 * 90 = 900 and 0 padding
- 10* 90 + 10*10 = 900 + 100 padding = 1000
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| timber |
| Posted: Thu Oct 19, 2017 1:56 pm Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| Quote: |
But I want the below result:
element1(10 characters)+element3(10 characters)+10 spaces = 30 in total
|
Please explain how the receiving application parses these messages. How does it know that characters 11..20 should be assigned to element 3 and not element2 (as would normally be the case)? |
|
| Back to top |
|
|
| andrewfemin |
| Posted: Thu Oct 19, 2017 6:09 pm Post subject: |
|
|
Acolyte
Joined: 26 Aug 2017
Posts: 54
|
| fjb_saper wrote: |
So taking this logic up to 10 records, do you want the output to be - 10 * 90 = 900 and 0 padding
- 10* 90 + 10*10 = 900 + 100 padding = 1000
|
It should be
- 10 * 90 = 900 and 0 padding
Last edited by andrewfemin on Thu Oct 19, 2017 7:44 pm; edited 1 time in total |
|
| Back to top |
|
|
| andrewfemin |
| Posted: Thu Oct 19, 2017 6:18 pm Post subject: |
|
|
Acolyte
Joined: 26 Aug 2017
Posts: 54
|
| timber wrote: |
| Please explain how the receiving application parses these messages. How does it know that characters 11..20 should be assigned to element 3 and not element2 (as would normally be the case)? |
I think the receiving application just displays the data in blocks of 10. Just that the logic written at the application has some limitation due to which it can receive data only in multiples of 100. I'm not sure though. |
|
| Back to top |
|
|
| timber |
| Posted: Sat Oct 21, 2017 5:08 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
I strongly suggest that you verify those assumptions becauses they sound wrong to me. I have dealt with many hundreds of fixed-length formats in my time, and I have never encountered a format where the position (character offset) was not important.
Do you have a written specification for this data format? If so, what does it say about this? |
|
| Back to top |
|
|
| andrewfemin |
| Posted: Mon Oct 23, 2017 2:23 am Post subject: |
|
|
Acolyte
Joined: 26 Aug 2017
Posts: 54
|
Your concern is understandable. This is new to me too and I double checked with the application team. This data is used for some displaying purposes and their order or missing elements do not matter. I know it sounds wrong. But it's a special case and this is required.
I've worked around it for now by adding a filler field and making it repeat as required in the esql code. But I'm curious to get this working only with dfdl. |
|
| Back to top |
|
|
|
|
