| Author |
Message
|
| KIT_INC |
 Posted: Thu May 30, 2019 9:15 am Post subject: How to handle {}:QFILE in XML tag ? |
 |
|
Knight
Joined: 25 Aug 2006
Posts: 589
|
I received a Transaction Start Event Monitoring message which contains
<wmb:complexContent wmb:elementName="{}:QFILE">
<{}:QFILE>
<Header>
<AppAppCreateDate>20190529</AppCreateDate>
<AppCreateTime>1414</AppCreateTime>
<LocNumber>1234</LocNumber>
<AppNumber>45678</AppNumber>
I do not have the original message. I was told that the flow developer specify Data Location $Body in monitoring properties and this is the event message generated.
I need to get the field LocNumber from it
The flow is just MQINPUT - COMPUTE - MQOUTPUT
My ESQL in COMPUTE
DECLARE wmb NAMESPACE 'http://www.ibm.com/xmlns/prod/websphere/messagebroker/6.1.0/monitoring/event';
DECLARE LN CHAR;
SET LN = InputRoot.XMLNSC.wmb:event.wmb:applicationData.wmb:complexContent.{}:QFILE.Header.LocNumber;
This does not set LN.
I use debug, I got PARSING at the MQINPUT node. The Input tree stop at
<wmb:complexContent wmb:elementName="{}:QFILE">
I think it is the {}:QFILE causing the error. IN another message I saw something similar except it is "{}:EFILE"
Is this something generated by the broker monitoring ? Did the flow developer did anything wrong which I should feedback ? What I have to do to get the field I want ? |
|
| Back to top |
|
 |
| timber |
| Posted: Thu May 30, 2019 1:24 pm Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
Have you tried putting quotes around the element name?
| Code: |
| SET LN = InputRoot.XMLNSC.wmb:event.wmb:applicationData.wmb:complexContent."{}:QFILE".Header.LocNumber; |
If that does not work then please supply the output of a Trace node. There is no other accurate way to show what the message tree looks like (the debugger does not display namespaces or field types) |
|
| Back to top |
|
|
| KIT_INC |
| Posted: Thu May 30, 2019 9:09 pm Post subject: |
|
|
Knight
Joined: 25 Aug 2006
Posts: 589
|
I wired 2 trace nodes
one (traceIn) between MQINPUT and COMPUTE
one (tracefail) to the failure terminal of the MQINPUT node.
The flow failed as soon as it picks up the message. I think this is because the trace node force it to do a full parsing.
tracein only shows
(0x01000000:Folder):XMLNSC = ( ['xmlnsc' : 0x1c7a2588d70]
after MQMD
tracefail shows the same
(0x01000000:Folder):XMLNSC = ( ['xmlnsc' : 0x1c7a2588d70]
I remove the trace node and took a user trace, it shows
2019-05-31 00:36:28.390532 18804 UserTrace BIP2537I: Node 'ETM_TO_CSV_FILE_MF.Compute': Executing statement ''SET LN = InputRoot.XMLNSC.wmb:event.wmb:applicationData.wmb:complexContent.{}:QFILE.Header.LocNumber;'' at ('.ETM_TO_CSV_FILE_MF_Compute.Main', '58.1').
2019-05-31 00:36:28.390648 18804 UserTrace BIP2543W: Node '': ('.ETM_TO_CSV_FILE_MF_Compute.Main', '58.14') : Failed to navigate to path element number '6' because it does not exist.
I did the debug again, the exceptionlist shows
Message
Properties
MQMD
XMLNSC
event
wmb http://www.ibm.com/xmlns/prod/websphere/messagebroker/6.1.0/monitoring/event
eventPointData
eventData
productVersion 10009
eventSchemaVersion 6.1.0.3
eventSourceAddress APP_CLAIM.RECEIVE.transaction.Start
eventIdentity
eventSequence
eventCorrelation
localTransactionId 72a7dbd4-7bba-11e9-98a6-a23518870000-13
parentTransactionId
globalTransactionId
messageFlowData
broker
executionGroup
messageFlow
node
nodeLabel APP_CLAIM.RECEIVE
nodeType ComIbmMQInputNode
detail APP_CLAIM.RECEIVE
applicationData
XML Parsing Errors have occurred
LocalEnvironment
So I am stuck again.
In {}:QFILE, the {} looks like namespace, so I tried
DECLARE {} NAMESPACE 'http://www.ibm.com';
But this gives me a Syntax error. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu May 30, 2019 11:27 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
So why did you not follow the directions Timber gave you? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| timber |
| Posted: Fri May 31, 2019 2:27 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
Please use [code] tags when posting trace outputs - it makes it much easier to read.
You said that there is an XML parsing error...but you have not quoted the error. No point in trying to fix the ESQL until you can parse the XML. Please find the error in the user trace output and post it here...in [code] tags. |
|
| Back to top |
|
|
| KIT_INC |
| Posted: Fri May 31, 2019 4:21 am Post subject: |
|
|
Knight
Joined: 25 Aug 2006
Posts: 589
|
| Quote: |
| So why did you not follow the directions Timber gave you? |
I did.
I wired the trace node. Sorry, I missed the use of quot in my last update.
'{}:QFILE' gave me syntax error.
"{}:QFILE" work in toolkit and allow me to deploy. The trace output says the same for {}:QFILE (without quote) or "{}:QFILE" (with double quote)
2019-05-31 08:04:17.799134 14272 UserTrace BIP2537I: Node 'ETM_TO_CSV_FILE_MF.Compute': Executing statement ''SET LN = InputRoot.XMLNSC.wmb:event.wmb:applicationData.wmb:complexContent.{}:QFILE.Header.LocNumber;'' at ('.ETM_TO_CSV_FILE_MF_Compute.Main', '58.1').
2019-05-31 08:04:17.799638 14272 UserTrace BIP2543W: Node '': ('.ETM_TO_CSV_FILE_MF_Compute.Main', '58.14') : Failed to navigate to path element number '6' because it does not exist.
Tracing does not trace anything happening inside the Input node.
The debug output just ends with parsing error and no further detail. I pasted it here using code
| Code: |
Message
Properties
MQMD
XMLNSC
event
wmb http://www.ibm.com/xmlns/prod/websphere/messagebroker/6.1.0/monitoring/event
eventPointData
eventData
productVersion 10009
eventSchemaVersion 6.1.0.3
eventSourceAddress APP_CLAIM.RECEIVE.transaction.Start
eventIdentity
eventSequence
eventCorrelation
localTransactionId 72a7dbd4-7bba-11e9-98a6-a23518870000-13
parentTransactionId
globalTransactionId
messageFlowData
broker
executionGroup
messageFlow
node
nodeLabel APP_CLAIM.RECEIVE
nodeType ComIbmMQInputNode
detail APP_CLAIM.RECEIVE
applicationData
XML Parsing Errors have occurred
LocalEnvironment
Environment
ExceptionList
|
|
|
| Back to top |
|
|
| Vitor |
| Posted: Fri May 31, 2019 4:35 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| KIT_INC wrote: |
'{}:QFILE' gave me syntax error.
"{}:QFILE" work in toolkit and allow me to deploy. The trace output says the same for {}:QFILE (without quote) or "{}:QFILE" (with double quote) |
For the record, a single quote indicates a string. A double quote is a short form of the EVAL function, which is what my most worthy associate was suggesting.
Inexperienced users should be aware that while "Hello Word" will evaluate to a string in the same way 'Hello World' does, the double quote evaluation costs a fortune in compute resources, will affect performance and should be used sparingly. Using it to define strings because you're used to a different programming language will not result in performant code.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri May 31, 2019 4:38 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Looks like your tree is not what you think it is...
| Code: |
2019-05-31 08:04:17.799134 14272 UserTrace BIP2537I: Node 'ETM_TO_CSV_FILE_MF.Compute': Executing statement ''SET LN = InputRoot.XMLNSC.wmb:event.wmb:applicationData.wmb:complexContent.{}:QFILE.Header.LocNumber;'' at ('.ETM_TO_CSV_FILE_MF_Compute.Main', '58.1').
2019-05-31 08:04:17.799638 14272 UserTrace BIP2543W: Node '': ('.ETM_TO_CSV_FILE_MF_Compute.Main', '58.14') : Failed to navigate to path element number '6' because it does not exist.
|
Failed to navigate to path element #6 because it does not exist...
You need to find out what the real path looks like.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| timber |
| Posted: Fri May 31, 2019 5:22 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
I said in my first post:
| Quote: |
| If that does not work then please supply the output of a Trace node. There is no other accurate way to show what the message tree looks like (the debugger does not display namespaces or field types) |
...but you have quoted the output of the debugger. Why? |
|
| Back to top |
|
|
| timber |
| Posted: Fri May 31, 2019 5:25 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| If the answer to my question is 'because there is a parsing error' then please do what I asked: quote the full text of the parsing error from the user trace (not the exception list, please) |
|
| Back to top |
|
|
| KIT_INC |
| Posted: Sun Jun 02, 2019 10:40 am Post subject: |
|
|
Knight
Joined: 25 Aug 2006
Posts: 589
|
| Quote: |
| please do what I asked: quote the full text of the parsing error from the user trace (not the exception list, please) |
I did what you ask already
I did not have the trace nodes output to user trace. I have them output to file and I have pasted their output in my previous post.
| Quote: |
tracein only shows
(0x01000000:Folder):XMLNSC = ( ['xmlnsc' : 0x1c7a2588d70]
after MQMD
tracefail shows the same
(0x01000000:Folder):XMLNSC = ( ['xmlnsc' : 0x1c7a2588d70]
|
As I said in my post, when I wired the trace node (tracein), the flow failed right away (i.e. nothing come out of out terminal of the MQINPUT node) and nothing shows in the user trace. I think this is because trace node force it to do a full parse instead of parse on demand. So it goes to the failure terminal where I have the trace node (tracefail) wired. Both trace output shows nothing under XMLNSC.
If output the trace node to user trace instead of to a file can show more, please let me know and I can do it.
The user trace output I show here is when there is no trace node and message comes out of the the out terminal of MQINPUT node into the compute node. When it looks for {} and cannot find it because the tree was never built and hence the "Failed to navigate to path element " error. My speculation, please correct me if I am wrong.
One more thing I found in my investigation is when I use XML validator , W3 school or XML spy, they both said "<{}:QFILE> " is an invalid element. So perhaps the broker is doing the right thing to fail the xml with parsing exception.
If that is the case, why is ETM monitoring giving us a XML with invalid element. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Jun 02, 2019 11:18 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Timber is not talking about the trace node output.
You need to set a user trace on the eg/flow and that is the output we need to help you there.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| KIT_INC |
| Posted: Sun Jun 02, 2019 2:09 pm Post subject: |
|
|
Knight
Joined: 25 Aug 2006
Posts: 589
|
I did run the user trace and pasted the output. User trace shows nothing on what the parser does.
Here is the trace up to the first declare statement in the compute node (i.e, Work inside MQINPUT is done)
| Code: |
Timestamps are formatted in local time, 240 minutes before GMT.
Trace written by version ; formatter version 10009 (build S1000-L170601.10408 on amd64_nt_4)
2019-05-31 08:04:11.177473 25444 UserTrace BIP4040I: The integration server ''TEST'' has processed a configuration message successfully.
A configuration message has been processed successfully. Any configuration changes have been made and stored persistently.
No user action required.
2019-05-31 08:04:11.178078 25444 Information BIP2154I: Integration server finished with Configuration message.
A command response will be sent to the integration node.
No user action required.
2019-05-31 08:04:11.178134 25444 UserTrace BIP11304I: The Parser of type ''XMLS'' has been deleted from address ''0x1c7a21eecb0''. This thread now has ''6'' cached parsers.
2019-05-31 08:04:11.178163 25444 UserTrace BIP11304I: The Parser of type ''MQROOT'' has been deleted from address ''0x1c7a258f5f0''. This thread now has ''5'' cached parsers.
2019-05-31 08:04:11.178179 25444 UserTrace BIP11304I: The Parser of type ''MQMD'' has been deleted from address ''0x1c7a9adfb50''. This thread now has ''4'' cached parsers.
2019-05-31 08:04:11.178209 25444 UserTrace BIP11305I: The Parser of type ''XMLS'' at address ''0x1c7a0808550'' has been reset ready for re-use. This thread has ''4'' cached parsers.
2019-05-31 08:04:11.178224 25444 UserTrace BIP11305I: The Parser of type ''MQROOT'' at address ''0x1c7a25902b0'' has been reset ready for re-use. This thread has ''4'' cached parsers.
2019-05-31 08:04:11.178237 25444 UserTrace BIP11305I: The Parser of type ''MQMD'' at address ''0x1c35b07b310'' has been reset ready for re-use. This thread has ''4'' cached parsers.
2019-05-31 08:04:11.178251 25444 UserTrace BIP11305I: The Parser of type ''Properties'' at address ''0x1c7a1be4be0'' has been reset ready for re-use. This thread has ''4'' cached parsers.
2019-05-31 08:04:11.178291 25444 UserTrace BIP11504I: Waiting for data from input node ''InputNode''.
A thread is waiting for data from input node ''InputNode'' in flow ''''.
2019-05-31 08:04:15.782423 11584 UserTrace BIP13025I: Connected to WebSphere MQ queue manager ''IIB10QM''.
Successfully connected to WebSphere MQ queue manager ''IIB10QM''.
2019-05-31 08:04:15.782754 11584 UserTrace BIP13024I: Disconnected from WebSphere MQ queue manager ''IIB10QM''.
Successfully disconnected from WebSphere MQ queue manager ''IIB10QM''.
2019-05-31 08:04:17.796230 14272 UserTrace BIP13026I: Read message from WebSphere MQ queue ''ETM_IN''.
Successfully read a message from WebSphere MQ queue ''ETM_IN'' on queue manager ''IIB10QM''.
2019-05-31 08:04:17.796328 14272 UserTrace BIP11501I: Received data from input node ''ETM_IN''.
The input node ''ETM_IN'' has received data and has propagated it to the message flow ''ETM_TO_CSV_FILE_MF''.
2019-05-31 08:04:17.796420 14272 UserTrace BIP11303I: A Parser of type ''MQROOT'' has been created at address ''0x1c7a258f5f0''. This thread now has ''1'' cached parsers.
2019-05-31 08:04:17.796522 14272 UserTrace BIP3907I: Message received and propagated to 'out' terminal of input node 'ETM_TO_CSV_FILE_MF.ETM_IN'.
2019-05-31 08:04:17.796568 14272 UserTrace BIP11303I: A Parser of type ''MQROOT'' has been created at address ''0x1c7a258f920''. This thread now has ''2'' cached parsers.
2019-05-31 08:04:17.796598 14272 UserTrace BIP11303I: A Parser of type ''Properties'' has been created at address ''0x1c7a1c04900''. This thread now has ''3'' cached parsers.
2019-05-31 08:04:17.796746 14272 UserTrace BIP11303I: A Parser of type ''MQROOT'' has been created at address ''0x1c7a0853a20''. This thread now has ''4'' cached parsers.
2019-05-31 08:04:17.796768 14272 UserTrace BIP11303I: A Parser of type ''MQROOT'' has been created at address ''0x1c7a0853d50''. This thread now has ''5'' cached parsers.
2019-05-31 08:04:17.796786 14272 UserTrace BIP11303I: A Parser of type ''MQROOT'' has been created at address ''0x1c7a08533c0''. This thread now has ''6'' cached parsers.
2019-05-31 08:04:17.796816 14272 UserTrace BIP11303I: A Parser of type ''Properties'' has been created at address ''0x1c7a1bfee70''. This thread now has ''7'' cached parsers.
2019-05-31 08:04:17.796840 14272 UserTrace BIP6060I: Node 'ETM_TO_CSV_FILE_MF.ETM_IN' used parser type ''Properties'' to process a portion of the incoming message of length '0' bytes beginning at offset '0'.
2019-05-31 08:04:17.796874 14272 UserTrace BIP11303I: A Parser of type ''MQMD'' has been created at address ''0x1c7a9ae2330''. This thread now has ''8'' cached parsers.
2019-05-31 08:04:17.796912 14272 UserTrace BIP6061I: Node 'ETM_TO_CSV_FILE_MF.ETM_IN' used parser type ''MQMD'' to process a portion of the incoming message of length '364' bytes beginning at offset '0'. The parser type was selected based on value ''MQHMD'' from the previous parser.
2019-05-31 08:04:17.796966 14272 UserTrace BIP11303I: A Parser of type ''XMLNSC'' has been created at address ''0x1c7a270d3b0''. This thread now has ''9'' cached parsers.
2019-05-31 08:04:17.797026 14272 UserTrace BIP6061I: Node 'ETM_TO_CSV_FILE_MF.ETM_IN' used parser type ''XMLNSC'' to process a portion of the incoming message of length '10737' bytes beginning at offset '364'. The parser type was selected based on value ''XMLNSC'' from the previous parser.
2019-05-31 08:04:17.797058 14272 UserTrace BIP11303I: A Parser of type ''MQROOT'' has been created at address ''0x1c7a08556d0''. This thread now has ''10'' cached parsers.
2019-05-31 08:04:17.797074 14272 UserTrace BIP11303I: A Parser of type ''MQROOT'' has been created at address ''0x1c7a08543b0''. This thread now has ''11'' cached parsers.
2019-05-31 08:04:17.797268 14272 UserTrace BIP2537I: Node 'ETM_TO_CSV_FILE_MF.Compute': Executing statement ''BEGIN ... END;'' at ('.ETM_TO_CSV_FILE_MF_Compute.Main', '2.2').
2019-05-31 08:04:17.797352 14272 UserTrace BIP2537I: Node 'ETM_TO_CSV_FILE_MF.Compute': Executing statement ''DECLARE wmb NAMESPACE 'http://www.ibm.com/xmlns/prod/websphere/messagebroker/6.1.0/monitoring/event';'' at ('.ETM_TO_CSV_FILE_MF_Compute.Main', '5.3'). |
Then is the navigation failure error which I have already pasted here before. |
|
| Back to top |
|
|
| timber |
| Posted: Sun Jun 02, 2019 2:19 pm Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
Sorry KIT_INC - the main problem is that you are not quoting *all* of the information that is required. I accept that you did quote the Trace node output correctly - but the message tree was empty because there was a parsing error.
When you quoted the exceptionList, you only quoted the outermost (i.e. the final) exception. Unfortunately, that's not helpful because it only says XML parsing errors have occurred. You need to expand the ExceptionList and look at the nested exceptions. You definitely will find a far more useful error if you do this. Please find that error and post it here.
When you quoted the user trace output, you stopped at the first ESQL statement. Not sure why, because IIB will not have done any parsing at that point. The execution of the ESQL is what triggers the parsing - unless you have changed the Parse Timing setting on the input node. You should find the parsing error a little further down the user trace. |
|
| Back to top |
|
|
| KIT_INC |
| Posted: Sun Jun 02, 2019 7:31 pm Post subject: |
|
|
Knight
Joined: 25 Aug 2006
Posts: 589
|
| Quote: |
| When you quoted the exceptionList, you only quoted the outermost (i.e. the final) exception |
unfortunately that is the entire exceptionlist and nothing else to expand.
| Quote: |
| When you quoted the user trace output, you stopped at the first ESQL statement. Not sure why, because IIB will not have done any parsing at that point. The execution of the ESQL is what triggers the parsing |
The reason I stopped at the first ESQL statement is to show that the message has exited the MQINPUT node and there is no useful information on what ,if any, the parser has done on the message.
From the first ESQL to the failing one there are of course many more entries on other ESQL statements. Unfortunately there are some business sensitive information that prevent me from posting the entire trace. I am working on a simplified case so that I can post the entire trace if needed.
I searched the entire trace, looking for parser error, there is none. The error happened on the execution of the ESQL was what I posted on May 31 which I posted here once more.
2019-05-31 08:04:17.799134 14272 UserTrace BIP2537I: Node 'ETM_TO_CSV_FILE_MF.Compute': Executing statement ''SET LN = InputRoot.XMLNSC.wmb:event.wmb:applicationData.wmb:complexContent.{}:QFILE.Header.LocNumber;'' at ('.ETM_TO_CSV_FILE_MF_Compute.Main', '58.1').
2019-05-31 08:04:17.799638 14272 UserTrace BIP2543W: Node '': ('.ETM_TO_CSV_FILE_MF_Compute.Main', '58.14') : Failed to navigate to path element number '6' because it does not exist. |
|
| Back to top |
|
|
|
|
