| Author |
Message
|
| mqsiuser |
 Posted: Wed Feb 29, 2012 1:24 am Post subject: |
 |
|
Yatiri
Joined: 15 Apr 2008
Posts: 637
Location: Germany
|
| mqjeff wrote: |
It is much easier to write a badly performant XPath expression than it is to write a badly performant ESQL expression.
And it's relatively easy to write a badly performant ESQL expression - just use [index]. |
I need to correct myself: For Java (JCN): Parts of xPath can be / might be harmful for your performance.
But that is not spectacular, since the same applies for famous 3 things in ESQL:
(1.) CARDINALITY, (2.) [index] and (3.) "myStructure.myparents....moreStuff...myelement"
@OP: Your requirement is "Use JDBC", thus it is "use Java"... be a bit aware of the xPath-expressions that you put in. Or: probably you just do not have considerable performance requirements (might also be a choice, but honestly... we kind of always want a good performance). Well... if you are a beginner you might be happy if your xPath works.
And now the ultimate tip for you: Use an xPath-Editor !!!, e.g. XPathBuilder to create (and test) xPath expressions before you put them into your (Java-)Code.
_________________
Just use REFERENCEs |
|
| Back to top |
|
 |
| kimbert |
| Posted: Wed Feb 29, 2012 3:18 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Here another good XPath tip. The easiest way to write an XPath expression is also the slowest. Instead of
| Code: |
| /root/Header/SenderDetails/Name |
use
| Code: |
| /root/Header[1]/SenderDetails[1]/Name[1] |
...otherwise the XPath engine will search for multiple matching elements at each level, which will require a complete search of the message tree.
Please note, though, that message broker will *never* parse the document multiple times just because you write an inefficient XPath expression. At worst, the document will be fully parsed when you didn't need that. |
|
| Back to top |
|
|
| mqsiuser |
| Posted: Thu Mar 01, 2012 8:42 am Post subject: |
|
|
Yatiri
Joined: 15 Apr 2008
Posts: 637
Location: Germany
|
| kimbert wrote: |
| Please note, though, that message broker will *never* parse the document multiple times just because you write an inefficient XPath expression. At worst, the document will be fully parsed when you didn't need that. |
On the interfaces that I write I kind of always need to fully parse the message. And still: I try to avoid ahead parsing, esp. the ones that are triggered by things that do not "really" need to access the data (basically I avoid CARDINALITY and use "while" instead of "for" loops). Why do I do that ?... Can I just turn off the on-demand-parsing?
And I am well aware of the difference between (re-)parsing and tree-walking (on parsed content). Is there a rough estimate on the (constant) factor that both differ ?
| mqjeff wrote: |
| side from that, a bad XPATH expression will ruin performance and potentially hog memory much faster and much worse than a bad ESQL expression |
I'd like to add that ESQL will more likely only harm your (run-)time performance, while some xPath-expressions can also harm memory (I guess you will quickly notice that when it happens and can do something about it then)
_________________
Just use REFERENCEs |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu Mar 01, 2012 12:46 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Looks as if we are heading for a record on the length of a single thread here...
| Quote: |
| On the interfaces that I write I kind of always need to fully parse the message. And still: I try to avoid ahead parsing, esp. the ones that are triggered by things that do not "really" need to access the data (basically I avoid CARDINALITY and use "while" instead of "for" loops). Why do I do that ?... Can I just turn off the on-demand-parsing? |
I don't know what you mean by 'ahead parsing', but hopefully the following facts will clear up any confusion:
a) each message is parsed at most once.
b) the cost of parsing the entire message is the same regardless of whether you use 'on-demand' or 'immediate' parsing.
c) if your flow always ends up accessing the entire input message then there is nothing that your flow can do to make the parsing any more ( or less ) expensive in CPU.
d) In some cases a carefully-designed message flow can avoid parsing the entire message, or parse it in chunks to minimise memory usage. In both cases, on-demand parsing is utilised.
So, the short answer is that in your case you could switch off on-demand parsing. But it will not make your message flows any faster or slower.
re: your last comment, badly-written ESQL can use a lot of memory too. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Thu Mar 01, 2012 6:34 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| mqjeff wrote: |
| rekarm01 wrote: |
| mqsiuser wrote: |
| I need to say it: xPath considered harmful (for performance) |
That remains to be seen. |
It is much easier to write a badly performant XPath expression than it is to write a badly performant ESQL expression.
And it's relatively easy to write a badly performant ESQL expression - just use [index]. |
Does that mean that ESQL should be considered harmful (for performance) too? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Mar 01, 2012 6:45 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| rekarm01 wrote: |
| mqjeff wrote: |
| rekarm01 wrote: |
| mqsiuser wrote: |
| I need to say it: xPath considered harmful (for performance) |
That remains to be seen. |
It is much easier to write a badly performant XPath expression than it is to write a badly performant ESQL expression.
And it's relatively easy to write a badly performant ESQL expression - just use [index]. |
Does that mean that ESQL should be considered harmful (for performance) too? |
It's relatively easy to avoid writing badly performant ESQL code. There are a couple of simple things to avoid, and then it runs reasonably well. You have to do a couple of other things, maybe, to ensure you have *good* performance, but you can avoid *bad* performance with not a lot of assistance.
I could be wrong, but I don't think it's quite that easy to avoid badly performant XSLT.
But I've never found XSLT to be terribly maintainable or readable, so I've avoided it for more reasons than just performance. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Thu Mar 08, 2012 1:55 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
The InfoCenter describes in more detail how on-demand parsing works.
| mqsiuser wrote: |
| I kind of always need to fully parse the message ... Can I just turn off the on-demand-parsing? |
It's cheaper to copy smaller trees from one msgflow node to another, rather than copying larger trees, so delaying full parsing until later in the message flow can still offer some performance benefit.
| mqsiuser wrote: |
| And still: I try to avoid ahead parsing, ... (basically I avoid CARDINALITY and use "while" instead of "for" loops). Why do I do that? |
Trying to delay the parsing of elements within a compute node has no effect on the cost to parse the elements. Using a WHILE loop instead of an equivalent FOR loop wouldn't delay parsing anyway. |
|
| Back to top |
|
|
| mqsiuser |
| Posted: Thu Mar 08, 2012 4:12 am Post subject: |
|
|
Yatiri
Joined: 15 Apr 2008
Posts: 637
Location: Germany
|
Thank you for providing the exact position in the infocenter... and also reflecting on the information provided there.
It is on by default. That must be enough hints  ... must have a reason. They also write quite a bit about it in the infocenter!
Imagine to process very large messages: 100.000 records/lines. The parser will immediately parse that... this will stop the world ... Imagine you have an error (that the code brings up... not a parser error!) in the first record  ... which performs better now ?! Note that likely this error in the first line is just the one that will throw the exception in your code... probably/likely/'just imagine' that e.g. all lines have (at least) one error. And now to make things totally worse: Your retry-count is 10.
When you program with the on-demand-parsing in mind... things don't get worse... but you also keep a door open to delete previous-in-siblings when you need to free up resources (at lets assume 2 days before code-freeze ). Though I must really add here, that you can also re-code anything (you just need a bit of time ) and also that I never needed to delete prev-in-siblings. But also: I do not really understand mem-management in Broker... and probably there is something which lifts up on-demand-parsing (you actually pointed to something... probably there is more).
And actual interface projects (at least for me) really do get complicated and you quickly require advanced things and can not just look at the introductory level (examples):
Processing "large" messages... you might consider this to be the long tail (of use cases). Thought: It seems to me that this is something that (SAP) systems often require. Though I really would always try and process like "records" (so 100.000 small messages) instead of a single big one (which includes 100.000 records)... but really this is not always (actually it is often for me) possible... since usually there are structural corrections required, which are based on "dependencies" between records. Also I would not dare to ask/tell one of our external principals/customers that we (want to) chop their message(s)... they want that we process what we receive as a whole.
And I guess I use WHILE just because FOR and CARDINALITY+[index] go too well with one another and WHILE and LASTMOVE also perfectly fit
The on demand parsing "applies to both the message body and the message headers"... even the MQMD and RFH2 headers are parsed on demand and adding to that (header-)fields are located at the top to allow e.g. quick identifying of the encoding to allow best performance for scenarios where e.g. only the codepage needs to be changed.
Does the parsing use its own (OS-)process and ESQL runs in another (the execution group's process)? Utilizing both (processes) at the same time (with on-demand-parsing) will then be better than just hitting into a significant dedicated parsing time (which would then be just utilizing one (the parsing) process).
And: Parsing and walking the (input-) message once (at the same time) is certainly better than parsing and then walking the input message (which would make 2 passes through the message (as opposed to one)).
_________________
Just use REFERENCEs |
|
| Back to top |
|
|
| sman123 |
| Posted: Wed Nov 23, 2016 11:23 am Post subject: ESQL to Java |
|
|
Newbie
Joined: 23 Nov 2016
Posts: 1
|
| Hi, I am looking for a way to programmatically convert an ESQL code into a Java code. Any help on this would be highly appreciated. Thanks. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Nov 23, 2016 11:50 am Post subject: Re: ESQL to Java |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| sman123 wrote: |
| Hi, I am looking for a way to programmatically convert an ESQL code into a Java code. Any help on this would be highly appreciated. Thanks. |
The only tool that I know of that can do this reliably and quickly and correctly is "a programmer".
There isn't an existing program that will do this, as far as I know.
In order to do this, you would need to write a parser for ESQL, and then a compiler to Java.
This is *extremely* complicated and messy.
If this is your idea, give up.
If this is management's idea, present them with a five hour lecture on writing parsers and compilers (and look for another job).
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Wed Nov 23, 2016 12:18 pm Post subject: Re: ESQL to Java |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| sman123 wrote: |
| Hi, I am looking for a way to programmatically convert an ESQL code into a Java code. Any help on this would be highly appreciated. Thanks. |
The general consensus here amongst us 
- IIB Is not a J2EE System.
- Java is the devil incarnate
- J2EE devs make rubbish IIB developers unless they get proper training.
- Java is a FOUR letter word
- Java is not the answer to anything.
Yes I am being rather frivolous but generally nothing can be gained from just changing the language (ESQL to JAVA).
There is nothing wrong with ESQL. It works and if it ain't broke, why fix it.
Yes, you probably have reasons for moving to Java but just converting the code won't work. It is a lot, lot more complicated than that. You have to be aware of all the other things that need changing in the IIB environment.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Nov 23, 2016 12:46 pm Post subject: Re: ESQL to Java |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
| smdavies99 wrote: |
Yes I am being rather frivolous but generally nothing can be gained from just changing the language (ESQL to JAVA).
There is nothing wrong with ESQL. It works and if it ain't broke, why fix it.
|
Well for some people (upstairs) the prospect of being able to only pay for the standard edition of IIB instead of paying for the advanced one is enough to make them drool and ask for the type of transformation we're hearing about here. 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Nov 23, 2016 1:30 pm Post subject: Re: ESQL to Java |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| fjb_saper wrote: |
| smdavies99 wrote: |
Yes I am being rather frivolous but generally nothing can be gained from just changing the language (ESQL to JAVA).
There is nothing wrong with ESQL. It works and if it ain't broke, why fix it.
|
Well for some people (upstairs) the prospect of being able to only pay for the standard edition of IIB instead of paying for the advanced one is enough to make them drool and ask for the type of transformation we're hearing about here. |
If they're only paying for the standard edition, then they shouldn't have any ESQL to convert...
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Nov 24, 2016 10:18 am Post subject: Re: ESQL to Java |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
| mqjeff wrote: |
| fjb_saper wrote: |
| smdavies99 wrote: |
Yes I am being rather frivolous but generally nothing can be gained from just changing the language (ESQL to JAVA).
There is nothing wrong with ESQL. It works and if it ain't broke, why fix it.
|
Well for some people (upstairs) the prospect of being able to only pay for the standard edition of IIB instead of paying for the advanced one is enough to make them drool and ask for the type of transformation we're hearing about here. |
If they're only paying for the standard edition, then they shouldn't have any ESQL to convert... |
The intention is to go from the full version to the standard to save costs...(reason for the drool...)
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Nov 28, 2016 8:59 am Post subject: Re: ESQL to Java |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| fjb_saper wrote: |
| The intention is to go from the full version to the standard to save costs...(reason for the drool...) |
Insert standard rant about 'saving cost' by 'paying less for software', without taking into account any other factors. "Look, we saved money, we paid less for software! We also paid 10x more for redevelopment and support and had to double our deadlines... but we reduced our capital expenditures!"
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
|
|
