| Author |
Message
|
| Ranodip Basu |
 Posted: Wed Apr 27, 2011 5:26 am Post subject: Parsing Err in TDS of CSV msg in AIX,But in Windows working |
 |
|
Apprentice
Joined: 23 Sep 2004
Posts: 48
|
Hi All,
I am using MQInput node to read a message which has multiple records with line feeder. The first record is header and 2nd record onwards are the body i.e. business records. All the fields in both header and body records are delimited by pipe separator ("|") and each of field are having double quotes. The message structure looks as in below
"header1"|"header2"|"header3"
"field1"|"field2"|"field3"|"field4"|"field5"
"field1"|"field2"|"field3"|"field4"|"field5"
"field1"|"field2"|"field3"|"field4"|"field5"
"field1"|"field2"|"field3"|"field4"|"field5"
"field1"|"field2"|"field3"|"field4"|"field5"
In the MRM message set I have used Messaging Standard as "CSV - Comma Separated Value" ,
Delimiter = |
Quote Character = "
Reserved Character = , <CR><LF>
The WMB version I am using is v6.1. If I execute the message flow in Windows XP version of WMB, then the message can be parsed succesfully.
In case I am using AIX version of WMB , then getting error as
(0x01000000:Name ):ParserException = (
(0x03000000:NameValue):File = '/build/S610_P/src/cpi/pwf/nxd/nxdworker.cpp' (CHARACTER)
(0x03000000:NameValue):Line = 462 (INTEGER)
(0x03000000:NameValue):Function = 'NXDWorker::parseNext' (CHARACTER)
(0x03000000:NameValue):Type = '' (CHARACTER)
(0x03000000:NameValue):Name = '' (CHARACTER)
(0x03000000:NameValue):Label = '' (CHARACTER)
(0x03000000:NameValue):Catalog = 'BIPv610' (CHARACTER)
(0x03000000:NameValue):Severity = 1 (INTEGER)
(0x03000000:NameValue):Number = 5421 (INTEGER)
(0x03000000:NameValue):Text = 'TDS General Error' (CHARACTER)
(0x01000000:Name ):Insert = (
(0x03000000:NameValue):Type = 5 (INTEGER)
(0x03000000:NameValue):Text = 'Message' (CHARACTER)
)
(0x01000000:Name ):Insert = (
(0x03000000:NameValue):Type = 5 (INTEGER)
(0x03000000:NameValue):Text = '/Message/Messaget_header/[SEQUENCE](1 of 2)/header(55 of unbounded)' (CHARACTER)
)
(0x01000000:Name ):Insert = (
(0x03000000:NameValue):Type = 2 (INTEGER)
(0x03000000:NameValue):Text = '914' (CHARACTER)
)
(0x01000000:Name ):ParserException = (
(0x03000000:NameValue):File = '/build/S610_P/src/cpi/pwf/nxd/nxdterminatingmarkupmatcher.cpp' (CHARACTER)
(0x03000000:NameValue):Line = 341 (INTEGER)
(0x03000000:NameValue):Function = 'NXDTerminatingMarkupMatcher::identifyMarkup' (CHARACTER)
(0x03000000:NameValue):Type = '' (CHARACTER)
(0x03000000:NameValue):Name = '' (CHARACTER)
(0x03000000:NameValue):Label = '' (CHARACTER)
(0x03000000:NameValue):Catalog = 'BIPv610' (CHARACTER)
(0x03000000:NameValue):Severity = 3 (INTEGER)
(0x03000000:NameValue):Number = 5627 (INTEGER)
(0x03000000:NameValue):Text = 'Quote character is not allowed in this position' (CHARACTER)
(0x01000000:Name ):Insert = (
(0x03000000:NameValue):Type = 5 (INTEGER)
(0x03000000:NameValue):Text = '"' (CHARACTER)
)
(0x01000000:Name ):Insert = (
(0x03000000:NameValue):Type = 2 (INTEGER)
(0x03000000:NameValue):Text = '926' (CHARACTER) |
|
| Back to top |
|
 |
| khudania |
| Posted: Wed Apr 27, 2011 5:37 am Post subject: |
|
|
Apprentice
Joined: 30 Nov 2004
Posts: 43
|
| Quote: |
In the MRM message set I have used Messaging Standard as "CSV - Comma Separated Value" ,
Delimiter = |
Quote Character = "
Reserved Character = , <CR><LF> |
Have you tried using """ instead of '' for Quote Character while configuring the message set for MRM. I feel this might work. |
|
| Back to top |
|
|
| Ranodip Basu |
| Posted: Wed Apr 27, 2011 7:11 am Post subject: |
|
|
Apprentice
Joined: 23 Sep 2004
Posts: 48
|
Hi Khudania,
Thanks for your reply. However using """ it is not working as because I have got the error message that the escape character (here quote character) should be of length 1 only.
(0x01000000:Name ):ParserException = (
(0x03000000:NameValue):File = '/build/S610_P/src/cpi/pwf/nxd/nxdworker.cpp' (CHARACTER)
(0x03000000:NameValue):Line = 415 (INTEGER)
(0x03000000:NameValue):Function = 'NXDWorker::parseFirst' (CHARACTER)
(0x03000000:NameValue):Type = '' (CHARACTER)
(0x03000000:NameValue):Name = '' (CHARACTER)
(0x03000000:NameValue):Label = '' (CHARACTER)
(0x03000000:NameValue):Catalog = 'BIPv610' (CHARACTER)
(0x03000000:NameValue):Severity = 3 (INTEGER)
(0x03000000:NameValue):Number = 5445 (INTEGER)
(0x03000000:NameValue):Text = 'TDS parsing error while starting parse (in parseFirst)' (CHARACTER)
(0x01000000:Name ):ParserException = (
(0x03000000:NameValue):File = '/build/S610_P/src/cpi/pwf/nxd/nxdterminatingmarkupmatcher.cpp' (CHARACTER)
(0x03000000:NameValue):Line = 899 (INTEGER)
(0x03000000:NameValue):Function = 'NXDTerminatingMarkupMatcher::setQuoteCharacter' (CHARACTER)
(0x03000000:NameValue):Type = '' (CHARACTER)
(0x03000000:NameValue):Name = '' (CHARACTER)
(0x03000000:NameValue):Label = '' (CHARACTER)
(0x03000000:NameValue):Catalog = 'BIPv610' (CHARACTER)
(0x03000000:NameValue):Severity = 3 (INTEGER)
(0x03000000:NameValue):Number = 5626 (INTEGER)
(0x03000000:NameValue):Text = 'Invalid length for escape character - escape character can only be one character in length' (CHARACTER)
(0x01000000:Name ):Insert = (
(0x03000000:NameValue):Type = 5 (INTEGER)
(0x03000000:NameValue):Text = '"' (CHARACTER) |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Apr 27, 2011 7:24 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Ranodip Basu wrote: |
| Thanks for your reply. However using """ it is not working as because I have got the error message that the escape character (here quote character) should be of length 1 only. |
Yes, it also wouldn't apply at all to MRM being mostly an XML construct.
Have you confirmed that the file on Unix has Unix line endings or that your message model accounts for both Windows line endings and unix line endings? |
|
| Back to top |
|
|
| khudania |
| Posted: Wed Apr 27, 2011 7:36 am Post subject: |
|
|
Apprentice
Joined: 30 Nov 2004
Posts: 43
|
Ranodip, I wonder why you got the following error
| Quote: |
'Invalid length for escape character - escape character can only be one character in length' (CHARACTER) |
Are you sure you selected(checked) the 'Quote Character' radio button and not the 'Escape Character' one, in the character data settings section ?
Please verify once. No harm in trying one more idea which doesn't work. |
|
| Back to top |
|
|
| Ranodip Basu |
| Posted: Wed Apr 27, 2011 8:49 am Post subject: |
|
|
Apprentice
Joined: 23 Sep 2004
Posts: 48
|
Hi Khudania,
I have checked the radio button for "Quote Character" in Character Data Setting section and not the "Escape Character". However both cases the error message is same , but the error codes are different 5626 and 5603 respectively.
(0x03000000:NameValue):Number = 5626 (INTEGER)
(0x03000000:NameValue):Text = 'Invalid length for escape character - escape character can only be one character in length' (CHARACTER)
(0x01000000:Name ):Insert = (
(0x03000000:NameValue):Type = 5 (INTEGER)
(0x03000000:NameValue):Text = '"' (CHARACTER)
(0x03000000:NameValue):Severity = 3 (INTEGER)
(0x03000000:NameValue):Number = 5603 (INTEGER)
(0x03000000:NameValue):Text = 'Invalid length for escape character - escape character can only be one character in length' (CHARACTER)
) |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Apr 27, 2011 8:52 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
You CANNOT use '"' in an MRM message definition. khudania was entirely wrong to suggest that and any further attempts to follow that advice is just wasting your time.
Please review my last comment about line endings. |
|
| Back to top |
|
|
| Ranodip Basu |
| Posted: Wed Apr 27, 2011 9:39 am Post subject: |
|
|
Apprentice
Joined: 23 Sep 2004
Posts: 48
|
Hi mqjeff,
Thanks for your input. The input message in AIX has HEX representation ofine feeder as 0A which is correct. One more thing I have found that if the fields are not encapsulated with " then the message is successfully parsed. The message should be
header1|header2|header3
field1|field2|field3|field4|field5
field1|field2|field3|field4|field5
field1|field2|field3|field4|field5
field1|field2|field3|field4|field5
field1|field2|field3|field4|field5
So the problem seems to be the quote (") in AIX.
Now coming to the question if the message model supports both windows line ending and unix line ending ,
I have defined the "Repeating Element Delimiter" in "Occurences" section of "Physical Properties" tab as <CR><LF> which is working fine in case of Windows. Could you please tell how to make then the message model also accounts for UNIX / AIX then |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Apr 27, 2011 9:41 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| My point is that the model might be expecting to see <cr><lf>, and when processed on uinx it is only finding <lf>, and then seeing another " on the next line, which it thinks is still part of the same record because it hasn't seen <cr><lf> |
|
| Back to top |
|
|
| flahunter |
| Posted: Wed Apr 27, 2011 7:09 pm Post subject: |
|
|
Acolyte
Joined: 30 Oct 2008
Posts: 62
|
| I agree, it should be <LF> issue |
|
| Back to top |
|
|
| khudania |
| Posted: Thu Apr 28, 2011 12:56 am Post subject: |
|
|
Apprentice
Joined: 30 Nov 2004
Posts: 43
|
| mqjeff wrote: |
| My point is that the model might be expecting to see <cr><lf>, and when processed on uinx it is only finding <lf>, and then seeing another " on the next line, which it thinks is still part of the same record because it hasn't seen <cr><lf> |
As per your suggestion, a feed with single line record should work. Ranodip Basu, I guess this one is sure worth trying. |
|
| Back to top |
|
|
| Gerd-in-ZA |
| Posted: Thu Apr 28, 2011 10:00 am Post subject: |
|
|
Novice
Joined: 13 Sep 2006
Posts: 14
Location: Johannesburg, South Africa
|
I may have an idea; please try adding the double quote (") to the reserved chars ...
_________________
-- Gerd -- |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu Apr 28, 2011 10:35 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
I think mqjeff has the correct diagnosis. The AIX line-end character is typically a single LF, whereas on Windows the usual line terminator is a carriage-return and line feed pair. The MRM parser does not allow you to be flexible about the line-end character, so you will need to use different message definitions for Windows / AIX.
What I'm trying to say is - don't experiment with quote characters or anything else until you have tried setting the Delimiter to <LF> instead of <CR><LF>. |
|
| Back to top |
|
|
| Gerd-in-ZA |
| Posted: Thu Apr 28, 2011 8:06 pm Post subject: |
|
|
Novice
Joined: 13 Sep 2006
Posts: 14
Location: Johannesburg, South Africa
|
I am trying to remember or find a reference - is there not (or should there not be) a "platform line end" generic denomination to cover this sort of situation?
Something that will be a <cr><lf> in Windows, a <lf> in Unix and a <nl> (x'15') on EBCDIC based systems?
Can't find the stupid thing. But ultimately I would like us to see one definiton that will work across platforms.
Adding the Quote to the reserved chars list in my mind might still be needed since it is a syntax element here. The MRM still tends to have surprises in store for me from time to time ..
(Most of the time I post here I know the answer; this one I am trying to help with the diagnosis because I don't and I am also trying to learn ...)
_________________
-- Gerd -- |
|
| Back to top |
|
|
| kimbert |
| Posted: Fri Apr 29, 2011 12:39 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
is there not (or should there not be) a "platform line end" generic denomination to cover this sort of situation?
Something that will be a <cr><lf> in Windows, a <lf> in Unix and a <nl> (x'15') on EBCDIC based systems?
|
Well...yes there should be, but there is not. Sorry to be the bearer of bad news.
There is some good news, though. It should be possible to handle both <LF> and <CR><LF> using a single message definition. Find the group that defines the <CR><LF>. Right-click this group and select 'Add sequence'. This will create a new, local sequence group under the existing group. Move all of the contents of the original group into the new, local sequence group. Set the delimiter ( or terminator, if that is how the <CR><LF> is defined ) on the new, local group to <LF>.
Make sure that the shorter of the two line terminators ( the <LF> ) is defined on the innermost of the two groups, otherwise this trick will not work.
There is a good chance that this will work, but it's not a guarantee. I know that the technique is sound, but I have never applied it to a message set with Messaging Standard set to 'CSV'. |
|
| Back to top |
|
|
|
|
