| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Issues while validating fixed width file |
« View previous topic :: View next topic » |
| Author |
Message
|
| burman.abhishek |
 Posted: Thu Jan 11, 2007 2:05 am Post subject: Issues while validating fixed width file |
 |
|
Apprentice
Joined: 23 Aug 2006
Posts: 37
|
I am using a message set using TDS format. The message structure is:
ComplexElement1(All Elements Delimited, Delimiter:CRLF)
-- Complex Element2 (Fixed Width, Max =-1, Repeating Element Identifier:CRLF)
--<Element1><Element2>......<ElementN>(Total Length:70)
The issue I'm facing is that, whenever a record of of length greater than 70 is encountered, the next record contains CRLF characters. Similarly, if a record is of total length less than 70, it shows '\r' or '\n' as the next characters. I am looking to validate each record and throw an exception whenever a record of length not equal to 70 is encountered.
_________________
With warm Regards,
Burman |
|
| Back to top |
|
 |
| kimbert |
| Posted: Thu Jan 11, 2007 3:42 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
hi,
That's an interesting problem. The TDS parser gives precedence to the innermost structure when it is parsing. You expected it to extract one line based on the delimiter ( CR/LF ) and then parse that line against the fixed-length structure. In fact, TDS does the opposite. It parses the fixed-length data first, and then assumes that the delimiter will be the next thing in the bitstream.
You can solve this in a couple of ways
A) Set Data Element Separation to 'Use Data Pattern' instead of 'All Element Delimited'. Set the data pattern for the structure to
B) Use two message definitions, and guide the parsing using ESQL. In your current message definition, replace the fixed-length structure with a single string field. Define a new message definition ( say, 'FLStruct' ) which defines the 70-character field. Use CREATE...PARSE to parse the inner definition against FLStruct message.
If you choose A), the TDS parser will pre-select the contents of each line and pass it to the inner fixed-length structure. If the line is > 70 characters, the remaining data will be put into the message tree as an anonymous syntax element. If you enable validation the MRM parser should throw an exception when it encounters the anonymous element because it is not in your model.
If the line is < 70 characters, the inner structure will run out of bitstream and the TDS parser will throw a parsing exception.
If you choose B), you will get BIP5288 ( bitstream too long ) when the line is too long and some other parsing exception when it is too short ( I can't say which - it probably depends on your model ).
I can't say which one will work best for you, because I don't know enough about your requirements. |
|
| Back to top |
|
|
| burman.abhishek |
| Posted: Fri Jan 12, 2007 6:23 am Post subject: |
|
|
Apprentice
Joined: 23 Aug 2006
Posts: 37
|
Hi,
Thanks for the quick reply.
I tried the solution (A) today, as I am looking to handle the irregular records before the ESQL. I tested successfully for a single -normal, longer or shorter record.
But, for more than one records, this method fails. I am keeping "Use Data Pattern" on the outer complex type and fixed length for inner complex type. It parses the first record successfully and does not handle the rest.
_________________
With warm Regards,
Burman |
|
| Back to top |
|
|
| kimbert |
| Posted: Fri Jan 12, 2007 6:37 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
It parses the first record successfully and does not handle the rest.
|
I think I know what the problem is ( although you do not quote any parsing exceptions, so I'm guessing  )
The pattern which I gave you will select all of the text before the next line feed. It will not include the line feed. The simple solution is to set Group Terminator to '<CR><LF>' on the outer complex type. |
|
| Back to top |
|
|
| burman.abhishek |
| Posted: Fri Jan 12, 2007 7:27 am Post subject: |
|
|
Apprentice
Joined: 23 Aug 2006
Posts: 37
|
I have tried putting <CR><LF>, but still its only picking up one record. I also tried keeping <Using Data pattern> at the inner complex level and then separating each record by then keeping a <Fixed Width> with group delimiter <CR><LF> at the top level. It still picks up just one record. I guess it just stops processing after finding the first carriage return/line feed.
_________________
With warm Regards,
Burman |
|
| Back to top |
|
|
| kimbert |
| Posted: Sat Jan 13, 2007 8:24 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Did you set maxOccurs to -1 on the outer element? |
|
| Back to top |
|
|
| mapa |
| Posted: Mon Jan 15, 2007 1:21 am Post subject: Question |
|
|
Master
Joined: 09 Aug 2001
Posts: 257
Location: Malmö, Sweden
|
I have a message set that can handle what I think you are describing.
A simplified example of data records below:
TAGDATA<CR><LF>
TAGDATA<CR><LF>
Real message contains header, trailer and different data records dependent upon value of TAG.
If setting validation option on MQInput node I get an exception both if the DATA is too short or too long. If not setting validation shorter records will pass without error.
This part of message is built using:
"Level 1": choice
* TDS Data element separation, length of tag 3
* Logical properties --> occurrences Min 1, max -1
"Level 2": Several local elements describing a record, TAGRecord:
* Type is local complexType
* Min occurrs 1, max 1
* TDS Tag = TAG
"Level 3": Local complex type
* Min occurrs 1, max 1
* TDS Data element separation= Fixed length
* TDS Group Terminator <CR><LF>
"Level 4": the fixed length data records
With this construct I loop on the Choice, and each iteration the choice contains only one record.
Please post an example of your message with neutral data in it if this help was not sufficient. I do believe I might find a way forward for you.
The real message looks almost like this, what I have described above is the "1101 to 1109" repeatingrecords. Section 10-11-11-98 is also looping within the 00 and 99 records.
00XXYYZZ<CR><LF>
10AABBCCDDEE<CR><LF>
1105VVBBNNNFFRR<CR><LF>
1107MMNNBBVVFF<CR><LF>
1105VVBBNNNFFRR<CR><LF>
1105VVBBNNNFFRR<CR><LF>
1106JJBBMMMFRR<CR><LF>
98EEDDCCBBAA<CR><LF>
99ZZYYXX<CR><LF>
Note, this file layout was forced upon us  |
|
| Back to top |
|
|
| burman.abhishek |
| Posted: Mon Jan 15, 2007 9:33 am Post subject: |
|
|
Apprentice
Joined: 23 Aug 2006
Posts: 37
|
The issue is resolved finally. My message set looks like this:
ComplexElement1(Fixed Width, MaxOccurs:-1)
-- Complex Element2 ('Use Data Pattern',)
--<Element1><Element2>......<Element7>
Data pattern for Elements: [^\r\n]{0,10}*
A record longer or shorter than the required 70 characters is failing during validation.
Thanks guys for all your help and suggestions.
_________________
With warm Regards,
Burman |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
