| Author |
Message
|
| wbi_telecom |
 Posted: Tue Jan 12, 2010 5:38 am Post subject: Unit of work |
 |
|
Disciple
Joined: 15 Feb 2006
Posts: 188
Location: Harrisburg, PA
|
Environment WMB 6.1 on Z linux
I have a flow that starts with a file input node. It reads the entire file as a single record and uses the XMLNSC parser. A copy of this message put on a queue for furthur processing. The queue on which the message is put is an Input queue for another flow.
Here is the problem. When I have an invalid XML message coming in the file, the first flow thorws an error once the message is put on the queue but the second flow still gets the message and starts processing it. IF I STOP THE SECOND FLOW, I DO NOT SEE THE MESSAGE ON THE QUEUE which means the message is rolled back after the exception in the first flow but what I don't understand is if the second flow is running how does it pick up the message before the unit of work of the first flow is over. I played with the transaction mode property in MQoutput node but I got the same result.
Can anyone please point me to the right direction?
Cheers, |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Jan 12, 2010 6:15 am Post subject: Re: Unit of work |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| wbi_telecom wrote: |
Can anyone please point me to the right direction?
|
Stop playing with the transaction property on the MQOuput node and get systematic. What combinations of valid/invalid, transactional/non-transactional yield what results?
How does the 1st flow react to the invalid XML? Does it rely on the default behaviour or do you have Failure terminals wired? I presume that you're parsing the message using XMLNSC to determine valid/invalid (and do you mean valid, or do you mean well formed here?) so what is the parsing set to? Is this done in the input node or do you have a Compute node downstream doing it?
Better information, better advice. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| nathanw |
| Posted: Tue Jan 12, 2010 6:21 am Post subject: |
|
|
Knight
Joined: 14 Jul 2004
Posts: 550
|
what happens in the first flow?
at what point do you have it being "duplicated" to the second flow, prior to any actions within the first flow or straightaway
if it should not be propogated to the second flow before any checks or edits etc in the first flow then ensure that the output to the queue is not carried out too early in the first flow.
_________________
Who is General Failure and why is he reading my hard drive?
Artificial Intelligence stands no chance against Natural Stupidity.
Only the User Trace Speaks The Truth  |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Jan 12, 2010 7:05 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
The second flow can't read the MQ message unless it has been committed by the first flow.
ergo, the write to the Queue must have been successful. In normal circumstances and as far as you have described it, that would end the particular unit of work related to the original input file.
Do you have a message model that you are parsing the input file against?
If so, are you performing any validation of the input against the model?
If I were you, I'd put a trace node before the MQOutput Node and trace on ${Root}. If you do this, please put any relevant bits of the trace output here.
This will help us to help you sort out the problem.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| wbi_telecom |
| Posted: Tue Jan 12, 2010 7:50 am Post subject: |
|
|
Disciple
Joined: 15 Feb 2006
Posts: 188
Location: Harrisburg, PA
|
The first flow sends the message right away to the second flow. As I said the exception is thrown when the message is put on the MQOutput in the first flow. I have an error handler connected to the Catch which captures the exception and exception says its an Invalid XML (yes its malformed. The XML is missing an end tag so the XML parser fails).
I have been systematic in my testing so far. Setting the transaction property to Yes or NO or Automatic yields same results. I am not validating the message against a schema. So when I get the message in Fileinput there is no exception in the flow. The parser fails to parse the message but the exception is thrown only after the message lands on the output node where bitstream is parsed.
Since the second flow gets the message, I assumed that the MQput by the first flow is successfull right? So I stopped the second flow and expected to see the XML message on the queue. But I do not see it which indicates that the first flow got the exception while doing the MQput. If this is true how is the second flow getting the message when its running? Why does the status of the second flow influencing the behavior of the MQput in the first flow?
The deubugger break point just before the MQoutput node says "XML parsing errors have occured".
Cheers, |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Jan 12, 2010 8:04 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| The deubugger break point just before the MQoutput node says "XML parsing errors have occured". |
If you are testing the error handling/transactional behavior of the flow, I would *strongly* advise you to disconnect the debugger. You can use user trace/Trace nodes to find out what's going on in your flow. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Jan 12, 2010 8:22 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| kimbert wrote: |
| Quote: |
| The deubugger break point just before the MQoutput node says "XML parsing errors have occured". |
If you are testing the error handling/transactional behavior of the flow, I would *strongly* advise you to disconnect the debugger. You can use user trace/Trace nodes to find out what's going on in your flow. |
This is getting to be your 'mantra' recently 
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Jan 12, 2010 8:25 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| This is getting to be your 'mantra' recently |
True!
Funnily enough, I'm quite a fan of the debugger...for the simple stuff. But as I've explained before, the debugger can mess about with the error handling behaviour of a flow, so it's best not to use it in this case. |
|
| Back to top |
|
|
| wbi_telecom |
| Posted: Tue Jan 12, 2010 10:04 am Post subject: |
|
|
Disciple
Joined: 15 Feb 2006
Posts: 188
Location: Harrisburg, PA
|
I ran a trace on both the flows and here it is. You can see that the MQput is successful in the first flow(Thread 5) after which there an exception thrown. The message that's put is picked up by the next flow(Thread 47).
| Code: |
2010-01-12 12:47:27.933918 5 UserTrace BIP3907I: Message received and propagated to 'out' terminal of input node 'soa.aes.wmb.xmlbreaker.SOA_XML_BREAKER.FileInput'.
2010-01-12 12:47:27.934770 5 UserTrace BIP6063I: A parser of type ''Properties'' was created on behalf of node 'soa.aes.wmb.xmlbreaker.SOA_XML_BREAKER.FileInput' to handle the input stream, beginning at offset '0'.
2010-01-12 12:47:27.934906 5 UserTrace BIP6064I: A parser of type ''XMLNSC'' was created on behalf of node 'soa.aes.wmb.xmlbreaker.SOA_XML_BREAKER.FileInput' to handle the input stream, beginning at offset '0'. The parser type was selected based on value ''XMLNSC'' from the previous parser.
2010-01-12 12:47:27.935570 5 UserTrace BIP4008I: Message propagated to '''out1''' terminal of node 'soa.aes.wmb.xmlbreaker.SOA_XML_BREAKER.Break_Large_XML'.
2010-01-12 12:47:27.935650 5 UserTrace BIP6065W: A stream source was asked to provide all of its data.
The input transport is capable of providing input data through a stream. This allows portions of the input data to be read as required, so using less memory than transports that retrieve all of the data.
Although the input transport can read the data through a stream, the data is being passed to a component in the broker that requires all of the data be read in.
If the input data is large, there might be increased memory usage within this execution group.
To help reduce memory usage, modify the message flow so that data provided by a stream is not passed to components in the broker that cannot process streams. In particular:
Avoid passing the data to parsers other than XMLNSC, MRM TDS and MRM CWF.
Avoid using ESQL functions, such as ASBITSTREAM and BITSTREAM, that request all of the data.
Avoid passing data to user-defined nodes and parsers.
Avoid passing the data to transports that do not support streams.
2010-01-12 12:47:27.938212 5 UserTrace BIP2638I: The MQ output node 'soa.aes.wmb.xmlbreaker.SOA_XML_BREAKER.SOA.COD.XML.OUT' attempted to write a message to queue ''SOA.COD.XML.OUT'' connected to queue manager ''''. The MQCC was '0' and the MQRC was '0'.
2010-01-12 12:47:27.938230 5 UserTrace BIP2622I: Message successfully output by output node 'soa.aes.wmb.xmlbreaker.SOA_XML_BREAKER.SOA.COD.XML.OUT' to queue ''SOA.COD.XML.OUT'' on queue manager ''''.
2010-01-12 12:47:27.938340 5 UserTrace BIP4008I: Message propagated to '''out2''' terminal of node 'soa.aes.wmb.xmlbreaker.SOA_XML_BREAKER.Break_Large_XML'.
2010-01-12 12:47:27.938802 5 UserTrace BIP4007I: Message propagated to 'out' terminal of node 'soa.aes.wmb.xmlbreaker.SOA_XML_BREAKER.COD_AUDIT.CREATE_AUDIT_MESSAGE'.
2010-01-12 12:47:27.940476 5 UserTrace BIP2638I: The MQ output node 'soa.aes.wmb.xmlbreaker.SOA_XML_BREAKER.COD_AUDIT.SOA.COD.TEMP.AUDIT' attempted to write a message to queue ''SOA.COD.TEMP.AUDIT'' connected to queue manager ''''. The MQCC was '0' and the MQRC was '0'.
2010-01-12 12:47:27.940492 5 UserTrace BIP2622I: Message successfully output by output node 'soa.aes.wmb.xmlbreaker.SOA_XML_BREAKER.COD_AUDIT.SOA.COD.TEMP.AUDIT' to queue ''SOA.COD.TEMP.AUDIT'' on queue manager ''''.
2010-01-12 12:47:27.985824 5 UserTrace BIP5004E: An XML parsing error ''The content of elements must consist of well-formed character data or markup.'' occurred on line 669 column 1 when parsing element ''/Root/XMLNSC/http://www.ed.gov/FSA/COD/2008/v3.0b:CommonRecord''. Internal error codes are '1519' and '2'.
This error was reported by the generic XML parser, and is usually the result of a badly formed XML message.
Check that the input XML message is a well-formed XML message that adheres to the XML specification. The line number and column number that are quoted in the message give the position where the parser discovered the problem. However, the actual error might be earlier in the message.
Other possible causes are:
1. A character that is not supported by XML occurs in the instance message data.
XML supports only a subset of control characters; therefore, ensure that no unsupported characters, such as X'00', appear in the document.
2. The Coded Character Set ID that is defined in the message header does not reflect the contents of the instance message.
If the XML document has an XML prologue, the WebSphere MQ CodedCharSetId should be consistent with the XML Encoding field.
3. A reserved XML character appears in the instance message data.
Characters that might be recognized as XML markup - for example, < and & - should be replaced with the corresponding XML entities - < and &.
2010-01-12 12:47:28.004650 5 UserTrace BIP5004E: An XML parsing error ''The content of elements must consist of well-formed character data or markup.'' occurred on line 669 column 1 when parsing element ''/Root/XMLNSC/http://www.ed.gov/FSA/COD/2008/v3.0b:CommonRecord''. Internal error codes are '1519' and '2'.
This error was reported by the generic XML parser, and is usually the result of a badly formed XML message.
Check that the input XML message is a well-formed XML message that adheres to the XML specification. The line number and column number that are quoted in the message give the position where the parser discovered the problem. However, the actual error might be earlier in the message.
Other possible causes are:
1. A character that is not supported by XML occurs in the instance message data.
XML supports only a subset of control characters; therefore, ensure that no unsupported characters, such as X'00', appear in the document.
2. The Coded Character Set ID that is defined in the message header does not reflect the contents of the instance message.
If the XML document has an XML prologue, the WebSphere MQ CodedCharSetId should be consistent with the XML Encoding field.
3. A reserved XML character appears in the instance message data.
Characters that might be recognized as XML markup - for example, < and & - should be replaced with the corresponding XML entities - < and &.
2010-01-12 12:47:28.005968 5 UserTrace BIP2231E: Error detected whilst processing a message in node 'soa.aes.wmb.xmlbreaker.SOA_XML_BREAKER.FileInput'.
2010-01-12 12:47:28.091640 47 UserTrace BIP2632I: Message received and propagated to 'out' terminal of MQ input node 'com.aes.wmb.codftp.COD_XML_FTP_FLOW.SOA.COD.XML.OUT'.
2010-01-12 12:47:28.091738 47 UserTrace BIP6060I: Parser type ''Properties'' created on behalf of node 'com.aes.wmb.codftp.COD_XML_FTP_FLOW.SOA.COD.XML.OUT' to handle portion of incoming message of length 0 bytes beginning at offset '0'.
2010-01-12 12:47:28.091766 47 UserTrace BIP6061I: Parser type ''MQMD'' created on behalf of node 'com.aes.wmb.codftp.COD_XML_FTP_FLOW.SOA.COD.XML.OUT' to handle portion of incoming message of length '364' bytes beginning at offset '0'. Parser type selected based on value ''MQHMD'' from previous parser.
2010-01-12 12:47:28.091830 47 UserTrace BIP6061I: Parser type ''XMLNSC'' created on behalf of node 'com.aes.wmb.codftp.COD_XML_FTP_FLOW.SOA.COD.XML.OUT' to handle portion of incoming message of length '255212' bytes beginning at offset '364'. Parser type selected based on value ''XMLNSC'' from previous parser.
2010-01-12 12:47:28.094510 47 UserTrace BIP4201I: Message propagated to out terminal from node 'com.aes.wmb.codftp.COD_XML_FTP_FLOW.XML to BLOB'.
A reset content descriptor node has received a message and is propagating it to any nodes connected to its out terminal.
No user action required.
2010-01-12 12:47:28.101286 47 UserTrace BIP4008I: Message propagated to '''out''' terminal of node 'com.aes.wmb.codftp.COD_XML_FTP_FLOW.INSERT_LINE_FEEDS'.
2010-01-12 12:47:28.102824 47 UserTrace BIP4201I: Message propagated to out terminal from node 'com.aes.wmb.codftp.COD_XML_FTP_FLOW.BLOB TO XML'.
A reset content descriptor node has received a message and is propagating it to any nodes connected to its out terminal.
No user action required.
2010-01-12 12:47:28.173412 47 UserTrace BIP4007I: Message propagated to 'out' terminal of node 'com.aes.wmb.codftp.COD_XML_FTP_FLOW.COD_AUDIT.CREATE_AUDIT_MESSAGE'.
2010-01-12 12:47:28.174418 47 UserTrace BIP2638I: The MQ output node 'com.aes.wmb.codftp.COD_XML_FTP_FLOW.COD_AUDIT.SOA.COD.TEMP.AUDIT' attempted to write a message to queue ''SOA.COD.TEMP.AUDIT'' connected to queue manager ''''. The MQCC was '0' and the MQRC was '0'.
2010-01-12 12:47:28.174436 47 UserTrace BIP2622I: Message successfully output by output node 'com.aes.wmb.codftp.COD_XML_FTP_FLOW.COD_AUDIT.SOA.COD.TEMP.AUDIT' to queue ''SOA.COD.TEMP.AUDIT'' on queue manager ''''.
2010-01-12 12:47:28.175888 47 UserTrace BIP4008I: Message propagated to '''out1''' terminal of node 'com.aes.wmb.codftp.COD_XML_FTP_FLOW.INSERT_LINE_FEEDS'.
2010-01-12 12:47:28.175916 47 UserTrace BIP4163I: Message propagated to the first terminal of the FlowOrder node 'com.aes.wmb.codftp.COD_XML_FTP_FLOW.FlowOrder'.
The FlowOrder node has received a message and has propagated it to the output terminal called first. Once the processing of the message down the first terminal has finished, the message will be propagated to the second terminal.
No user action required.
2010-01-12 12:47:28.178448 47 UserTrace BIP4164I: Message propagated to the second terminal of the FlowOrder node 'com.aes.wmb.codftp.COD_XML_FTP_FLOW.FlowOrder'.
The FlowOrder node has finished processing a message down the first terminal and has propagated it to the second terminal.
No user action required.
Threads encountered in this trace:
47 5
|
|
|
| Back to top |
|
|
| Luke |
| Posted: Tue Jan 12, 2010 10:05 pm Post subject: |
|
|
Centurion
Joined: 10 Nov 2008
Posts: 128
Location: UK
|
| wbi_telecom wrote: |
| I have an error handler connected to the Catch which captures the exception |
Hi
What does your error handler do after it's captured the exception? Do you throw again to rollback the transaction?
Sounds unusual that you stop the second flow and the message doesn't appear on the queue. Dumb question - have you checked the open input count of that queue after you stop the flow? Always good to make sure something else (e.g. another instance of that flow somewhere) isn't reading your messages.
Regarding Kimbert's suggestion for removing debugger, have you re-tested the various transaction settings without debugger?
Cheers |
|
| Back to top |
|
|
|
|
