| Author |
Message
|
| adars1 |
 Posted: Fri Apr 01, 2011 2:40 pm Post subject: parsing error during TDS of csv file |
 |
|
Apprentice
Joined: 13 Jun 2010
Posts: 39
|
Hi,
Iam using csv file for fileinput node to parse the incoming file.
each line is treated as one record and each record is delimited by ,
if my file input line is like:
87044444,KJUL,0,2,"have been sent package on (03.15 03.16), campaign LL"
then it parses fine.
if the input line is like:
87023333,IJKL,0,2,"have been sent package on (03.15 03.16), campaign LL",
Iam getting error:
<Error><FlowName>Inbound</FlowName><TimeStamp>2011-04-01 16:58:05.994420</Ti
meStamp><ErrorDetail><RecoverableException><File>/build/S610_P/src/DataFlowEngin
e/ImbTraceNode.cpp</File><Line>341</Line><Function>ImbTraceNode::evaluate</Funct
ion><Type>ComIbmTraceNode</Type><Name>Inbound#FCMComposite_1_14</Name><Label
>Inbound.Trace1</Label><Catalog>BIPv610</Catalog><Severity>3</Severity><Numb
er>2230</Number><Text>Caught exception and rethrowing</Text><ParserException><Fi
le>/build/S610_P/src/MTI/MTIforBroker/MtiImbParser2/MtiImbParser.cpp</File><Line
>509</Line><Function>MtiImbParser::parseFirstChild</Function><Type>ComIbmFileInp
utNode</Type><Name>Inbound#FCMComposite_1_1</Name><Label>Inbound.CCHGI
N</Label><Catalog>BIPv610</Catalog><Severity>3</Severity><Number>5285</Number><T
ext>ImbRecoverableException caught from worker->parseNext.</Text><Insert><Typ
e>5</Type><Text>ICUMsgSet</Text></Insert><Insert><Type>2</Type><Text>1</Text></I
nsert><Insert><Type>5</Type><Text>Text1</Text></Insert><Insert><Type>5</Type><Te
xt>/msg_INBOUND/NEWVAL</Text></Insert><ParserException><File>/build/S610_P/s
rc/cpi/pwf/nxd/nxdworker.cpp</File><Line>462</Line><Function>NXDWorker::parseNex
t</Function><Type></Type><Name></Name><Label></Label><Catalog>BIPv610</Catalog><
Severity>1</Severity><Number>5421</Number><Text>TDS General Error</Text><Insert>
<Type>5</Type><Text>msg_INBOUND</Text></Insert><Insert><Type>5</Type><Text>/
msg_INBOUND/NOTE</Text></Insert><Insert><Type>2</Type><Text>21</Text></Inser
t><ParserException><File>/build/S610_P/src/cpi/pwf/nxd/nxdterminatingmarkupmatch
er.cpp</File><Line>341</Line><Function>NXDTerminatingMarkupMatcher::identifyMark
up</Function><Type></Type><Name></Name><Label></Label><Catalog>BIPv610</Catalog>
<Severity>3</Severity><Number>5627</Number><Text>Quote character is not allowed
in this position</Text><Insert><Type>5</Type><Text>"</Text></Insert><Insert
><Type>2</Type><Text>136</Text></Insert></ParserException></ParserException></Pa
rserException></RecoverableException></ErrorDetail></Error>
only difference between 2 lines is , at the end of 2nd line. |
|
| Back to top |
|
 |
| kimbert |
| Posted: Fri Apr 01, 2011 3:26 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
a) You have not described your message model. Are you using the model provided in the CSV sample?
b) Did you set a delimiter on the FileInput node? Or are you passing the entire file to the message flow, and letting the TDS parser split it into records? |
|
| Back to top |
|
|
| adars1 |
| Posted: Mon Apr 04, 2011 5:16 am Post subject: |
|
|
Apprentice
Joined: 13 Jun 2010
Posts: 39
|
- using the MRM parser and defined the msgset with TDS delimiter ,
- splitting the file at file input node through end of line |
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Apr 04, 2011 5:45 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| using the MRM parser and defined the msgset with TDS delimiter , |
Your input file uses quote characters to allow commas and line feeds within a field. Does your MRM message set cope with that?
| Quote: |
| splitting the file at file input node through end of line |
Please can you take a debug-level user trace and post the relevant part here. You will need to run the following commands in the following order:
- mqsichangetrace
- mqsireadlog
- mqsiformatlog
One possibility is that you are actually passing the entire file to the TDS parser. So maybe the first thing you should do is check that you have specified the line feed character correctly in the FileInput node. |
|
| Back to top |
|
|
| adars1 |
| Posted: Mon Apr 04, 2011 6:36 am Post subject: |
|
|
Apprentice
Joined: 13 Jun 2010
Posts: 39
|
- in input file node, iam splitting records by delimited dos or unix line end.
- after this node Iam using ResetContentDescriptor node to parse the each record with MRM parser and using the msgset properties to split the record using delimiter '.
-msgset msg format :
Data Element seperation: All Elements Delimited
Delimiter: , |
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Apr 04, 2011 6:44 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| Does your MRM message set cope with that? |
I'll answer my own question: I think you're OK, because you would not have got error message ( BIP5627 ) if you had not set up your quoting style to follow CSV rules.
So...what about my other questions? |
|
| Back to top |
|
|
| adars1 |
| Posted: Fri Apr 08, 2011 9:14 am Post subject: |
|
|
Apprentice
Joined: 13 Jun 2010
Posts: 39
|
Andalso the msgset complex type physical property 'Suppress Absent Element Delimiters' is 'Never'. IBM support suggested to use 'End of Type', but it will create other problems of not validating the previous missing fields.
I did take trace debug, but it is big enough to post it here. |
|
| Back to top |
|
|
|
|
