| Author |
Message
|
| timclark |
 Posted: Mon Sep 23, 2013 10:11 am Post subject: Interface Performance |
 |
|
Newbie
Joined: 19 Sep 2013
Posts: 4
|
Hi, I am fairly new to Message Broker/Integration Bus and looking for some best practices or recommendations around emitting data from a message flow for consumption by another Broker based application with regard to performance. I see three possibilities:
1. Explicit write to a queue that represents the interface to the second application
2. Publish/Subscribe using the Publication built-in node to target queue through subscription destination
3. Built-in monitoring events for a selected node to target queue through subscription destination
4. Something else?
Assuming the messages to the queue will not be persistent and the data would be written outside the transaction, what are the performance implications for each approach?
I might assume that #2 & #3 might result in more overall CPU utilization but do those interfaces get more of the processing out of the message flow critical path? Since this is intended to a high-performance, high volume application, will one provide better message through-put over the others for the primary flow?
Thanks in advance for any insight. |
|
| Back to top |
|
 |
| lancelotlinc |
| Posted: Mon Sep 23, 2013 10:16 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
Hi Tim -welcome.
IMHO The most efficient way to transfer a reasonable amount of data transactionally is through Web Services (unless the destination is running in the same site, in which case GlobalCache may be faster). The next most efficient is via queues. Pub/sub is less efficient unless you have many destinations.
In any event, unless you are doing more than 10,000 transactions per second, you won't have to worry about CPU utilization.
Lance
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Mon Sep 23, 2013 11:22 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| Quote: |
IMHO The most efficient way to transfer a reasonable amount of data transactionally is through Web Services
|
Apart from the fact that the OP has not mentioned transactionality, I can't imagine that SOAP will be faster than MQ for passing data between flows.
In my experience SOAP is slow when compared to MQ especially if you don't use persistent messages.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Sep 23, 2013 11:35 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
@smdavies I didn't say 'SOAP', I said Web Services, which can also include HttpRequest node with no SOAP marshaling.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Sep 23, 2013 12:17 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Why not simply gather accounting and statistics? |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Sep 24, 2013 12:33 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
3. would be a little perverse. Monitoring events publish information to monitoring applications and transaction monitoring systems. They are not intended to be used to carry the primary output of the message flow.
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
|
| timclark |
| Posted: Tue Sep 24, 2013 8:58 am Post subject: |
|
|
Newbie
Joined: 19 Sep 2013
Posts: 4
|
Thanks for the responses so far...
I tried to avoid characterizing the secondary application as a monitoring system in order to focus on the mechanics of getting the data from one place to another with as little impact as possible on the primary flow. But, that is essentially what it is, the primary flow will be able to detect 'events' that the secondary flow would be interested in and would need to get details and data from that event to the secondary flow that would be processed independently and asynchronously (one way).
I guess I was trying to understand if any one of those approaches caused Message Broker/Integration Bus to offload some of those mechanics on a parallel thread that would provide the best though-put. Each of the options I outlined offers us different advantages in terms of development, flexibility and maintenance and I wanted to add this performance element to help us decide the best approach for our situation...or am I unnecessarily splitting hairs on a system that will likely be less than 10k trans/sec with message sizes less than 64k?
Any thoughts? |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Sep 24, 2013 9:02 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
|
| Back to top |
|
|
| timclark |
| Posted: Tue Sep 24, 2013 11:51 am Post subject: |
|
|
Newbie
Joined: 19 Sep 2013
Posts: 4
|
I am not after statistics, these are events specific to our primary application and can contain large amounts of data directed to our secondary applications.
Thanks! |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Sep 25, 2013 12:43 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
OK - that changes things then. You are trying to emit lightweight messages to a 'monitoring' application ( or at least, an application that has requirements very similar to those of a monitoring application ).
In that case, 3. is the best option. You can ask for the monitoring events to be emitted outside of the main transaction if you want them to be received regardless of whether the main transaction succeeded.
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
|
| timclark |
| Posted: Wed Sep 25, 2013 3:58 am Post subject: |
|
|
Newbie
Joined: 19 Sep 2013
Posts: 4
|
| Is there a rule of thumb about a practical limit to the amount of data pushed out through a monitoring event? It looks like the base WMB Event message is around 1KB. How much more data can be reasonably added? |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Sep 25, 2013 4:06 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Not really - it depends on your requirements. I've seen huge amounts of data put into 'monitoring' messages when they're used for transaction monitoring/auditing.
I think it would be worth supplying a few more details, so that we can give more specific advice.
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
|
|
|
