| Author |
Message
|
| Lillian |
 Posted: Thu Sep 09, 2004 4:20 am Post subject: MQSI PERFORMANCE( parsing) |
 |
|
Centurion
Joined: 15 Apr 2002
Posts: 102
|
Hi
We have a flow that processes a 800 kb message. the performance is fine up until we parse the message using the MRM CWF on the compute node.. We need to parse in order to do the mapping. The time spent in this compute node is 2 seconds, which is unacceptable to the business users.
We have isolated the code from the time spent in the node.
any ideas on improving this time --is it possible???) |
|
| Back to top |
|
 |
| zpat |
| Posted: Thu Sep 09, 2004 4:34 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| Version of MQSI being used? |
|
| Back to top |
|
|
| Lillian |
| Posted: Thu Sep 09, 2004 5:01 am Post subject: |
|
|
Centurion
Joined: 15 Apr 2002
Posts: 102
|
| We are using wmqi 2.1 wth CSD06. Running the Broker on Sun Solaris |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Sep 09, 2004 5:55 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
There are techniques for optimizing execution time of ESQL.
If you are writing loops, it is always better to use references rather than indexes. This can make a big improvement in performance.
If you can, avoid parsing as much data as possible. Only access the fields you need to access, rather than the whole tree.
Support Pack IP04 has more information.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| shanson |
| Posted: Thu Sep 09, 2004 5:58 am Post subject: |
|
|
Partisan
Joined: 17 Oct 2003
Posts: 344
Location: IBM Hursley
|
You'll find that parsing/writing messages is the dominant factor in all but the most trivial of message flows. Some suggestions, some of which may be useful depending on your processing:
1) It's quicker if you are not validating messages, so only do so if you have a business need, don't just do it by default.
2) MRM uses partial ('on-demand') parsing, so as much of the message is parsed as is needed to satisfy the ESQL requests. If you are only routing a message then make sure routing data is at the start of the message. If you are transforming just part of the message and ignoring the rest, same rule applies.
As you have only pinned the problem down to a node, it is possible your ESQL might not be as efficient as it could be too. |
|
| Back to top |
|
|
| shanson |
| Posted: Thu Sep 09, 2004 6:05 am Post subject: |
|
|
Partisan
Joined: 17 Oct 2003
Posts: 344
Location: IBM Hursley
|
| Please be aware that IP04 is correct in what it says but is way out of date - 11 May 2001. I've highlighted this and will post back if we can get some more up-to-date info available. |
|
| Back to top |
|
|
| Lillian |
| Posted: Thu Sep 09, 2004 6:18 am Post subject: |
|
|
Centurion
Joined: 15 Apr 2002
Posts: 102
|
The reason I say I have removed/isolated the code from the time taken to process the message is this: The outgoing format has default definitions on the elements and all we do now is assign one element of the message (to access the MRM message definition.) In so doing, we are not mapping each element with incoming data but accepting the defaults constraintsalready defined. The output message is of the correct (expected)length and has the default values in data.
Because of the above we deduce that it is the creation of this message ( parsing) that is taking time and not the code ( ps: we have implemented the above suggestions to improve the ESQL) |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Sep 09, 2004 6:50 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| shanson wrote: |
| Please be aware that IP04 is correct in what it says but is way out of date - 11 May 2001. I've highlighted this and will post back if we can get some more up-to-date info available. |
Is it that out of date for 2.1, CSD6?
Sure, v5... but 2.1?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| shanson |
| Posted: Thu Sep 09, 2004 6:55 am Post subject: |
|
|
Partisan
Joined: 17 Oct 2003
Posts: 344
Location: IBM Hursley
|
| It is out of date for 2.1 as well as 5.0. |
|
| Back to top |
|
|
| shanson |
| Posted: Thu Sep 09, 2004 6:58 am Post subject: |
|
|
Partisan
Joined: 17 Oct 2003
Posts: 344
Location: IBM Hursley
|
| I can't picture exactly what your ESQL is doing, please append it. It's not clear to me that your message is being parsed at all. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Sep 09, 2004 11:46 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| What about using a message format that only describes (in detail at least) the section of the message you are interested in, or do you need to access every field? |
|
| Back to top |
|
|
| Lillian |
| Posted: Fri Sep 10, 2004 12:44 am Post subject: |
|
|
Centurion
Joined: 15 Apr 2002
Posts: 102
|
The extent of the mapping depends on the number of "contracts" that the in coming message has, so the repeat count has be set to max (300) and set defaults if the contracts are less.
The total number (max) of elements that get mapped is 70221 ( including repeat counts or loops)
To answer your question, we need to access all the elements as defined in MRM(CWF)... Or do we???
As long as the output message has a consistent size and all required data the receiving app is happy. |
|
| Back to top |
|
|
| zpat |
| Posted: Fri Sep 10, 2004 12:51 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
I think the issue here is not the size, but the number of elements defined.
Unfortunately MQSI can often sequentially search the message tree so that large trees (in terms of numbers of elements) will make the processing take exponentially longer.
If you can reduce the element count this will help - for example unless you are transforming each field individually why not define them as fewer, longer, strings - you can still copy all the data to the output message if it is unchanged from the input message. |
|
| Back to top |
|
|
| kimbert |
| Posted: Fri Sep 10, 2004 3:27 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Another suggestion, in case it helps...
| Quote: |
| The extent of the mapping depends on the number of "contracts" that the in coming message has, so the repeat count has be set to max (300) and set defaults if the contracts are less. |
You could put a 'repeat count' field at the start of the message, and make the repeating structure refer to it. That way, if your incoming message only contains 150 'contracts' you would only parse and write half as many fields. Of course, this assumes that the receiving application can handle a message which contains a repeat reference. |
|
| Back to top |
|
|
| PGoodhart |
| Posted: Fri Sep 10, 2004 4:36 am Post subject: |
|
|
Master
Joined: 17 Jun 2004
Posts: 278
Location: Harrisburg PA
|
If you can't get it to go faster through optimization, then you need to through more processor and memory at the problem to met your performance needs. If that is even a possiblity. Two seconds really isn't that much time, even from a realtime perspective. That's about what I get on a top end dual processor AIX box with 2gigs of ram on similarly sized files that require parsing. The realtime rule of thumb is 7 second response times.
We put our routing information/instructions into the header to remove the need to parse the data (at all) and those flows take effectively no time (sub-second).
_________________
Patrick Goodhart
MQ Admin/Web Developer/Consultant
WebSphere Application Server Admin |
|
| Back to top |
|
|
|
|
