| Author |
Message
|
| mqxplorer |
 Posted: Wed Nov 02, 2011 1:35 pm Post subject: XPath expressions impact on WMB message flows performance |
 |
|
Master
Joined: 22 Jun 2009
Posts: 206
|
While I was going through a document for WMB recommended practices, I came across the below sentence which made me ask this question in the forum.
| Code: |
| Although XPath is a powerful statement from a development point of view it can have an unforeseen performance impact in that it will force a full parse of the input message. By default it will retrieve all instances of the search argument. |
The concern which I have now after reading this sentence is - we are using the event monitoing faeture in WMB V7. We have a flow which processes the emiited xml messages and inserts some key data and payload into database. In order to have the nodes emit the xml documents (which has key fields and payload), we need to specify the XPath expression for the required key field and $Body for the payload emission for the node in the event monitoring tab.
As we are specifying $Body for payload emission, as per the above sentence for XPath expression, the whole message will be parsed.. Is that right? If so, event monitoing with XPath expression specification oevrrides the functionality of partial parsing??? Then it would be a major performance impact.
When this parsing happens, I guess, the broker does not parse the whole message for every XPath expression specified for different key field data in event monitoing...???
I read in a document ....
| Code: |
| Elements that are already parsed are not reparsed |
hopefully, this is true, otherwise event monitoirng adds a lot of performance overhead.
It would be really great, if someone can clarify my doubts.
Thanks in advance.
Thanks
mqxplorer |
|
| Back to top |
|
 |
| Vitor |
| Posted: Thu Nov 03, 2011 4:30 am Post subject: Re: XPath expressions impact on WMB message flows performanc |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqxplorer wrote: |
| we need to specify the XPath expression for the required key field and $Body for the payload emission for the node in the event monitoring tab. |
Why not tick the little box on the event monitoring & have broker emit the payload as a base64 string? Very efficient.
| mqxplorer wrote: |
When this parsing happens, I guess, the broker does not parse the whole message for every XPath expression specified for different key field data in event monitoing...???
I read in a document ....
| Code: |
| Elements that are already parsed are not reparsed |
|
Which document did you read that in, and are you sure it was refering to XPath not ESQL?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqxplorer |
| Posted: Thu Nov 03, 2011 11:34 am Post subject: Re: XPath expressions impact on WMB message flows performanc |
|
|
Master
Joined: 22 Jun 2009
Posts: 206
|
| Vitor wrote: |
Why not tick the little box on the event monitoring & have broker emit the payload as a base64 string? Very efficient.
|
We have explored this option sometime back, however, based on the input I got from the thread http://www.mqseries.net/phpBB2/viewtopic.php?t=55888&postdays=0&postorder=asc&start=0 I started sometime back, we decided to use $Body. The other thing we have is, we capture some key fileds of the incoming data which needs XPath expression specification in the event monitoring tab. This also causes full message parisng overriding the default 'On Demand/Partial Parsing??
| Vitor wrote: |
Which document did you read that in, and are you sure it was refering to XPath not ESQL? |
I am not 100% sure, however, it might be referring ESQL.. The sentce I have seen is...
| Code: |
Typically, the Broker parses elements up to and including the required field
• Elements that are already parsed are not reparsed |
Coming back to my original question...
Event monitoing with XPath expression specification oevrrides the functionality of partial parsing???
I guess, I can take it as YES based on your answer??
Thanks
mqxplorer |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Nov 03, 2011 11:55 am Post subject: Re: XPath expressions impact on WMB message flows performanc |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqxplorer wrote: |
| I am not 100% sure, however, it might be referring ESQL.. |
I think so.
| mqxplorer wrote: |
Typically, the Broker parses elements up to and including the required field
• Elements that are already parsed are not reparsed |
That's a description of broker's handling of the message tree.
| mqxplorer wrote: |
Typically, the Broker parses elements Event monitoing with XPath expression specification oevrrides the functionality of partial parsing???
|
I don't think it overrides it per se. I think any XPath evaluator requires a fully parsed XML document including the one broker is using. I'll leave it to other better qualified than me to confirm or deny this, and to comment on if this behaviour is in any way configurable.
| mqxplorer wrote: |
| we decided to use $Body |
You might find this interesting and/or useful.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| kimbert |
| Posted: Fri Nov 04, 2011 1:55 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
XPath is just another way to access the message tree. It does not ( necessarily ) force a complete parse of the message. It does what ESQL or Java or PHP or the Mapping node would do - it asks for the parts of the message tree that it needs. If those parts are not present then it drives the parser to produce them.
So...no problem so far.
However, it is very easy to write inefficient XPath. IF you are SURE that there is EXACTLY one occurrence of your element in the message tree THEN you can make your XPath more efficient by adding 'predicates' to each step in the path. So instead of writing
| Code: |
| $body/root/header/record/field1 |
you should write
| Code: |
| $body/root/header[1]/record[1]/field1[1] |
This allows the XPath processor to stop much earlier and it may mean that the your flow parses a lot less data - but that depends very much on what the remainder of the message flow is doing. |
|
| Back to top |
|
|
| mqxplorer |
| Posted: Mon Nov 07, 2011 11:16 am Post subject: |
|
|
Master
Joined: 22 Jun 2009
Posts: 206
|
Hi Kimbert,
Thanks for your response. I got the answer for my first conecern which is parsing of the XPath expression specification for key fields...like $Body/Element/Element[1]/elementA, however, the second question/concern I have is.. what happens if we specify $Body in the event monitoing tab of the node for payload emission? Does this override the partial parsing of the node?
Thanks
mqxplorer |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Nov 07, 2011 12:07 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| mqxplorer wrote: |
Hi Kimbert,
Thanks for your response. I got the answer for my first conecern which is parsing of the XPath expression specification for key fields...like $Body/Element/Element[1]/elementA, however, the second question/concern I have is.. what happens if we specify $Body in the event monitoing tab of the node for payload emission? Does this override the partial parsing of the node?
Thanks
mqxplorer |
If you don't change it it should just pass the input stream to the output stream. Kimbert will correct me if my assumption here is wrong...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Nov 07, 2011 1:32 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| Kimbert will correct me if my assumption here is wrong |
I'd love to, but in this case no correction is necessary  |
|
| Back to top |
|
|
| mqxplorer |
| Posted: Mon Nov 07, 2011 2:21 pm Post subject: |
|
|
Master
Joined: 22 Jun 2009
Posts: 206
|
| fjb_saper wrote: |
If you don't change it it should just pass the input stream to the output stream. Kimbert will correct me if my assumption here is wrong... |
I am little bit confused here. When you say, "if you don't change it"..does it mean if the message is not altered by the message flow in the subsequent nodes? So, if the message is altered by the subsequent nodes the message will be fully parsed overriding the Partial Parsing??
We have flows where we get an input request onto MQInput node which has $Body specified on the Event Monitoring tab for payload emission and the compuet node after input node transforms the message to an application specific (a web service) xml format and calls the web service to send this update. In this scenario does the MQInput node fully parses the incoming bitstream by overriding the 'Partial Parsing' option set on the 'Parse timing' dropdown list?
May be I am asking the same question again and again but I want to undwerstand the parser behaviour little more in depth and it implications on message flow performance.
Thanks in advance.
Thanks
mqxplorer |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Nov 07, 2011 2:49 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| mqxplorer wrote: |
| fjb_saper wrote: |
If you don't change it it should just pass the input stream to the output stream. Kimbert will correct me if my assumption here is wrong... |
I am little bit confused here. When you say, "if you don't change it"..does it mean if the message is not altered by the message flow in the subsequent nodes? So, if the message is altered by the subsequent nodes the message will be fully parsed overriding the Partial Parsing?? |
My reference was only valid for the Event monitoring.
| mqxplorer wrote: |
We have flows where we get an input request onto MQInput node which has $Body specified on the Event Monitoring tab for payload emission and the compuet node after input node transforms the message to an application specific (a web service) xml format and calls the web service to send this update. In this scenario does the MQInput node fully parses the incoming bitstream by overriding the 'Partial Parsing' option set on the 'Parse timing' dropdown list?
May be I am asking the same question again and again but I want to undwerstand the parser behaviour little more in depth and it implications on message flow performance.
Thanks in advance.
Thanks
mqxplorer |
You were concerned about the overhead on event monitoring.
Looking at the setup if you just use ${Body} which means no change to the bitstream, you should encurr the same overhead as when you specify
| Code: |
| SET OutputBody = InputBody; |
Now your message is being manipulated and used for your application's need. You need to realize that with each manipulation comes the cost of that manipulation: parsing on demand until the needed input is parsed and serialization on output, writing the new message.
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqxplorer |
| Posted: Tue Nov 08, 2011 11:23 am Post subject: |
|
|
Master
Joined: 22 Jun 2009
Posts: 206
|
| fjb_saper wrote: |
You were concerned about the overhead on event monitoring.
Looking at the setup if you just use ${Body} which means no change to the bitstream, you should encurr the same overhead as when you specify
| Code: |
| SET OutputBody = InputBody; |
|
May be I am not getting what you mean by "no change" but I am still confused with it. When you say "no change".. what exactly you mean...? I am using the $Body to tell the broker runtime to emit the payload as an xml document.
| Code: |
Looking at the setup if you just use ${Body} which means no change to the bitstream, you should encurr the same overhead as when you specify
[code]SET OutputBody = InputBody;[/code]
|
Do you mean to say that the Input node does the message tree copy?? My understing is .. the input node converts the message bit stream into a structured object which is message tree and this process is called parsing.
Would it be possible for you to explain me little bit more about this? I am sorry to ask again but I just want to be clear about the performnace overhead of event monitoring. I am not able to find the relavent info in info center.
Thanks in advance...
Thanks
mqxplorer |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Nov 08, 2011 11:32 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Well when you use
| Code: |
| set OutputBody = InputBody; |
there is no real need for parsing as the output bitstream is just going to be the same as the input bitstream, right? (No change of properties and ccsid).
So as long as output bitstream and input bitstream are the same why would you want to have the overhead of parsing?...
Like I said this is different when we are talking about a transformation or mapping.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
