| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Latency comparison: WTX or JCN or custom node. |
« View previous topic :: View next topic » |
| Author |
Message
|
| lancelotlinc |
 Posted: Mon Jun 28, 2010 5:25 am Post subject: Latency comparison: WTX or JCN or custom node. |
 |
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
Good morning to all.
Ok, so we've had an opportunity to implement and run some performance comparisons and we're not seeing the results we need from WTX. Just tossing this topic out to see if others have similar experiences and what some potential resolutions may be.
We're on WMB Version: 7.0.0 Build id: 7.0.0-IFix-20100308_1835 and WTX Version: 8.3.0.2 Build id: 23 . We have had IBM Global Services onsite all along the way helping us make good design decisions and we feel fairly confident that the design is solid.
The application intending to be replaced by ESB has a reliable round-trip latency of between 30 and 40 milliseconds. While we understand that ESB may be somewhat slower, it's current benchmarks for the exact same work is around 1,640 milliseconds.
Our basic flow is this (asynchronous request-reply pattern): MQInput -> WTX (blob to XML) -> MQOutput [for the request]; and MQInput -> WTX (XML to blob) -> MQOutput [for the reply].
Of the 1,640 millisecond average latency, 1,638 is spent in WTX node. Management is now starting to balk at the ESB concept as part of the sales pitch was overall better performance. But 1,640 is not in the ballpark with 30 milliseconds of the legacy application. In order for us to be successful, we would need ESB to come in around several hundred milliseconds at most (our target is 100 ms for this flow).
In various PMRs we have raised with IBM, we have sent the whole code in on those WTX tickets, and they have solved the PMRs successfully, noting that our WTX translations appear complex. They did not however mention any way to simplify or streamline the WTX operations.
We are wondering if maybe hand-coding a JCN or custom coding a C node might yield better performance than WTX? We dont see a performance advantage to using WTX. The only business justification for using WTX might be to have a library of legacy support formats at the ready, but we don't see any reason to use it for performance. We have source code that could port easily, wondering why IBM GS pointed us to WTX rather than port our existing C source code for performance reasons?
The 15 year old legacy application was written in C and runs on Solaris.
Any insight would be quite helpful. We are doing a simple thing, not complicated at all. It should be quite fast or at least comparable to the legacy application.
Thoughts?
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
 |
| kimbert |
| Posted: Mon Jun 28, 2010 10:55 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| We are doing a simple thing, not complicated at all. It should be quite fast or at least comparable to the legacy application. |
Can you describe it for us? It's pretty difficult to advise without knowing at least a little about what type of transformations the message flow is performing.
As a minimum, we'll need to know:
- input format ( XML / standard non-XML / custom non-XML )
- output format
- message sizes
- type and complexity of the transformation logic |
|
| Back to top |
|
|
| kirank |
| Posted: Mon Jun 28, 2010 2:13 pm Post subject: |
|
|
Centurion
Joined: 10 Oct 2002
Posts: 136
Location: California
|
I have worked with WTX node in 6.1 and I would not necessarily use WTX node over compute node for performance improvement. In fact I had create a PMR for performance reason as WTX node could not handle large maps. We had created a WTX map using WMB toolkit and for large maps WTX was failing an generating abend files.
WTX works well if you are using Industry Packs such as HIPAA pack for healthcare as it has readily available means of parsing messages and you don't have to build Message Sets.
A compute node with ESQL might be the option for you but I will need more details. You mentioned your input message is BLOB. Is this message being parsed using WTX Type tree or you are using BLOB because WTX node wants it? What format is input message?
Regards
Kiran Kanetkar |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Jun 29, 2010 9:10 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
Hi Kimbert -
Yes, I can give some additional detail. The input blob is an EBCDIC Big Endian variable length data structure with some header information and one or more detail information leafs. At the end of each leaf is a two-byte data length value that indicates if and how many bytes following comprise the next leaf.
We've had difficulty with the WTX node on a certain condition where the WTX cursor within the structure gets confused. A PMR is open and we are working on that issue. If we hard code a certain flag, we can avoid that issue, so we are working around that until such time as we get patch from IBM on it.
The performance numbers indicated in my original post are from Windows and our target platform is AIX. On AIX, our performance is 400 ms round trip, which is still a little outside the ballpark (but much better than Windows).
Some concepts I would like to stress: we wholeheartedly believe the WMB and WTX products are great products and fully support their market niche. A lot of hard work by many great people with superb vision at IBM have produced these fantastic products and are extremely well implemented. We have received WORLD CLASS technical support from all we have contacted at IBM during our implementation. Absolutely hands-down the best technical support I have ever received in my 30 year career. My post here is to solicit potential work arounds, because we may not have an application that best plays to the strengths of these products.
Hope this gives more insight. Thanks for your interest.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Jun 29, 2010 10:47 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| Yes, I can give some additional detail. The input blob is an EBCDIC Big Endian variable length data structure with some header information and one or more detail information leafs. At the end of each leaf is a two-byte data length value that indicates if and how many bytes following comprise the next leaf. |
The MRM TDS parser could probably parse that format, but it's impossible to be certain without a complete and accurate description of the format, with examples. No criticism implied, btw - just pointing out why I can't be sure.
Why do I mention this? The usual recommendation is to use the WMB native facilities unless they are not up to the job - it keeps performance up and complexity down. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Jun 29, 2010 11:07 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
We originally implemented a pilot using MRM TDS and we ended up in a JCN because we could not efficiently determine how to parse the variable number of leafs which could each be variable length. We successfully used the JCN to hand-parse the incoming blobs.
Not able to publicly post the specifics, but did include all in our PMRs.
Here is a generalized example:
Msg 1 - Header 246 bytes, leaf 1 is 42 bytes, leaf 2 is 109 bytes, leaf 3 22 bytes. Byte 245 and 246 indicate 42. Byte 287 and 288 indicate 109.
Msg 2 could be same as msg 1 but different number of leafs and different lengths for the leafs.
Messages have any number of leafs. Some messages could have ten leafs, some messages could have two leafs, some could have none.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Jun 29, 2010 11:33 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
It's probably easier to think of the leafs as starting with a byte-count, than with them ending with the byte-count of the NEXT segment.
Or at least, that may lead to a more sensible model.
There are a couple of specific cases that TDS can't handle: bit level field models and dynamically determined delimiters or tag values.
That is, TDS and MRM in general can't handle fields that are less than a full byte long.
It also can't handle things where a portion of the message tells you what characters or byte values are going to be used to separate other portions of the message.
The later is the main reason that MIME can't be handled in MRM and required it's own domain... (or at least.. the Broker Archtiect at the time decided to build a MIME domain rather than extend MRM to handle dynamic delimiters).
If you don't have either of those, then MRM/TDS can likely your message format, and it should be at least as performant, if not more so, than calling out to WTX to do parsing and transformation and then having to cross back into Broker to do routing and communications.
That said, Broker is never going to be as performant as a bespoke special purpose widget for the same task. But that's true of everything - no general purpose tool is ever going to be as performant as a special purpose widget. But a general purpose tool can be more *flexible*.
You should be able to get Broker to be performant enough, with tuning and optimization and scaling. |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Jun 29, 2010 12:39 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
There are a couple of specific cases that TDS can't handle: bit level field models and dynamically determined delimiters or tag values.
|
In case anyone is reading this in future, one small clarification: the MRM TDS parser can parse certain industry-standard formats like HL7, ACORD, X12 in which the data specifies its own delimiters. What it lacks is *generalized* support for formats that specify the delimiters in the data. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
