| Author |
Message
|
| MrSmith |
 Posted: Tue Oct 11, 2011 5:55 am Post subject: MRM Delimited Fixed Length issue |
 |
|
Master
Joined: 20 Mar 2008
Posts: 215
|
Forum search has produced some postings but not trawled through them all tbh :
Data:
H1111122222333334444455555
12345ABSCDFG00000
ABCDF123457813456
02356857GDHEJKLYSHRJKYPOT
Data definition:
H type - Fixed Length elements containing H - 11111 - 22222 - 33333 - 44444 - 55555 end of line is <LF>
Subsequent lines (no tag delimiting pattern avail) but have fixed length fields of 12345 - ABSC- DFG0 - 0000
so string(5) - string(4) - string(4) - string(4) again ending with <LF>
parses fine with all the subsequent line values present but the the last 2 strings are non mandatory. Have set the minoccurs to 0 if the last 2 elements do not exist then the parse fails. Having scoured the help I thought maybe that i could get round this with using default values i.e if doesnt exist just fill it up and the structure is preserved - this is true but only on the write it would appear but i want to know is there something that says if those values are there fill with this on incoming parse not an outgoing. Read also that it seems to ignore minoccurs on fixed length complexes when the parent is delimited or maybe i didnt read that right
Sorry if this is confusing it is always easier to explain verbally than in writing, so if unclear please state what would help define this better.
Thanks |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Tue Oct 11, 2011 10:29 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Have you tried tagged delimited with delimiter = <LF>? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| MrSmith |
| Posted: Wed Oct 12, 2011 1:04 am Post subject: |
|
|
Master
Joined: 20 Mar 2008
Posts: 215
|
fjb_saper
Cant use tag delim on detail records there is no constant tag, the data is just for effect, each line is different.
Yes to using delimiting though, that is exactly what is delimiting it all now and as i say that works fine i get header and detail (subsequent) records parsed, but that is providing that the detail records have values in all the proposed fields (i.e the last 2 x string (4)) but they are optional and markd so in the message set but when the incoming "record" is not fully populated for the last 2 fields it fails parsing.
example:
this works fine
11111222233334444<LF>
this doesnt
111112222<LF>
hence my original posting - if I write this then it will fill the last 2 fileds with defaults if so configured but it doesnt do it for an incoming parse it would seem, is this correct in which case the app will have to supply those values for the fixed length.
Thx
Last edited by MrSmith on Wed Oct 12, 2011 1:31 am; edited 1 time in total |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Oct 12, 2011 1:22 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
AHAA... this means you don't have fixed length delimited, you have variable length delimited and you need to make some fields in the record as optional...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Oct 12, 2011 1:31 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
You cannot use Tagged Delimited in this scenario - the fields themselves are fixed-length fields. It is the *structure* that is delimited.
So...I think your best option is:
- outermost complex type has Data Element Separation set to Tagged Delimited. Delimiter is set to <LF>
- the complex type for each record has Data Element Separation set to 'Use Data Pattern'.
- Each field in each record has the 'Data Pattern' property set. For fixed-length numeric fields it should be \d{5} ( assuming that the length is 5 digits ). |
|
| Back to top |
|
|
| MrSmith |
| Posted: Wed Oct 12, 2011 1:42 am Post subject: |
|
|
Master
Joined: 20 Mar 2008
Posts: 215
|
kimbert
Am following what your saying in the config but not understanding, the Data Pattern isnt that only going to tell the parser to what "record type" it belongs bit like a tag delim? Within my ESQL I still need to be able to reference those details record fields so the complex structure would be made up of say ordernumber (5), depot(4) but the optional is the weeknumber(4), targetdepot(4) say, my understanding of your suggestion would be that the parser would "define" the detail record based on the data pattern but how will that allow the last 2 fields to be optional and still parse?
If I am being thick here thats cos I am 
_________________
-------- *
“Outside of a dog, a book is man's best friend. Inside of a dog it's too dark to read.” |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Oct 12, 2011 2:10 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
No, you're not being thick, but I'm still right
The missing piece of information, is this rule:
If Data Element Separation=Use Data Pattern AND a field is optional AND the data pattern match returns zero length THEN the MRM TDS parser assumes that the optional field is missing from the data. |
|
| Back to top |
|
|
| MrSmith |
| Posted: Wed Oct 12, 2011 2:40 am Post subject: |
|
|
Master
Joined: 20 Mar 2008
Posts: 215
|
OK nice one will give that a go - thanks.
_________________
-------- *
“Outside of a dog, a book is man's best friend. Inside of a dog it's too dark to read.” |
|
| Back to top |
|
|
| MrSmith |
| Posted: Thu Oct 13, 2011 3:24 am Post subject: |
|
|
Master
Joined: 20 Mar 2008
Posts: 215
|
Kimbert
Tried yur suggestion marvellous outcome  except one thing even though the detail record has a mandatory on the first 2 fields using data pattern parsed a line that had only the first manadatory fields and therefore i assume that using data pattern it negates any fixed length validation, it is circumvented i suppose by coaleasce the field in ESQL but is this what ypu expected that mandatories are in effect ignored?
I tried to read the "rules" applied to FL in TD etc but they a bit vague tbh
Thx
_________________
-------- *
“Outside of a dog, a book is man's best friend. Inside of a dog it's too dark to read.” |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu Oct 13, 2011 3:39 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
You have a couple of options available to you:
a) Split each record into two sequence groups. The mandatory fields go into the first group, which has DES='Fixed Length'. The optional fields go into the second group which has DES='Use Data Pattern'.
b) Set Validation to 'Content and value' so that you get a validation exception if a mandatory field is missing from the data. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Oct 13, 2011 4:13 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
I might be tempted to approach this by modelling the entire contents as children of the header.
Then you could use tagged/delimited to match, and use a group delimiter to match the internal line endings.
I don't say, at this time, whether this would work. But it's how I might tend to approach it. |
|
| Back to top |
|
|
| MrSmith |
| Posted: Thu Oct 13, 2011 5:30 am Post subject: |
|
|
Master
Joined: 20 Mar 2008
Posts: 215
|
Kimbert - thanks for the extra input will try it out - had thought maybe I could have placed a regex on the mandatory fields to insure they had a min of 4 digits of 0-9 and same for the other mandatory which it did of sorts but then left the other orphaned values as just that even tho the parsed passed. I see where you coming from will try that (its more a chllenge now as have coded ESQL to COALESCE for field which doesnt appear in the Detail record) but just want to get it to "validate per se" itself.
mqjeff - That was the approach I originally tried (one of many) but it wouldnt parse on partial lines, i presume this is because although the record is fixed length it is tag delimited at the structure level and therefore optional element validation is ignored?
_________________
-------- *
“Outside of a dog, a book is man's best friend. Inside of a dog it's too dark to read.” |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Oct 13, 2011 5:58 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Well, again, I didn't say it would work.
But I'd think that you'd only need the outer record to be tagged-delimited, and the remainder just delimited - again, using the line endings as group terminators for the non-header record.
Of course that might also require that the file/message ends with two line endings, one to terminate the group that matches the last line and one to terminate the record. |
|
| Back to top |
|
|
|
|
