| Author |
Message
|
| deepnair |
 Posted: Tue Feb 28, 2012 9:21 am Post subject: Subflows Vs Individual Mainflows (hops) |
 |
|
Apprentice
Joined: 07 Feb 2012
Posts: 35
|
Hi,
I have a design question as below
Lets say there are Four functions A, B, C, D which needs to be implemented in sequential order i.e. A first then B, then C and then D. To implement this I see 2 ways
Approach 1
Write four different flows and connect them as F1 --- > F2 --- > F3 --- > F4 thus a total of 3 MQ hops to achieve it
Approach 2
Write one main flow and have A, B, C, D as subflows i.e. 4 subflows thus no MQ hops
Which of these two will give me better TPS (transactions per second) ??
The first one has more hops but I think its good because it doesnt make my EG thread hungry by holding any single thread longer as each F1, F2 , F3, F4 will have their own instances.
In the second apprach, there are no MQ hops but it will hold the thread for a longer time i.e. till A, B, C, D functions are all complete.
Thoughts ?
Thanks,
Deep Nair |
|
| Back to top |
|
 |
| lancelotlinc |
| Posted: Tue Feb 28, 2012 9:24 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
Best TPS is Approach 2. Adding MQ to the latency increases your latency around 13 msec per queue (you suggest 3 queues, total increase would be 39 msecs).
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| paintpot |
| Posted: Tue Feb 28, 2012 9:27 am Post subject: |
|
|
Centurion
Joined: 19 Sep 2005
Posts: 112
Location: UK
|
Transactionality? Is it valid to complete in multiple units of work?
Time to complete each 'chunk'. If any of the parts take a long time to complete, you will hold less resources / have better throughput if you split it up with MQ. You can then balance the work / latency by having multiple copies at the right point.
Partial reuse? Will other applications want to make use of the 'bits' you suggest in diffferent ways?
As usual - it depends... |
|
| Back to top |
|
|
| deepnair |
| Posted: Tue Feb 28, 2012 9:31 am Post subject: |
|
|
Apprentice
Joined: 07 Feb 2012
Posts: 35
|
| Quote: |
| Best TPS is Approach 2. |
So you are suggesting MQ latency is worse than thread latency ?
If I have 20 transactions hitting a second, and my # of instances of the flow is defined as 10, then the next 10 will wait for threads to be available. |
|
| Back to top |
|
|
| deepnair |
| Posted: Tue Feb 28, 2012 9:33 am Post subject: |
|
|
Apprentice
Joined: 07 Feb 2012
Posts: 35
|
| Quote: |
| If any of the parts take a long time to complete, you will hold less resources / have better throughput if you split it up with MQ |
This is what I think is better approach i.e. approach 1... |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Feb 28, 2012 9:36 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| deepnair wrote: |
| Quote: |
| Best TPS is Approach 2. |
So you are suggesting MQ latency is worse than thread latency ?
If I have 20 transactions hitting a second, and my # of instances of the flow is defined as 10, then the next 10 will wait for threads to be available. |
I told the doctor, it hurts when I do that. He said, well don't do that.
If you expect 20 concurrent transactions, you should load 19 additional instances. This is not rocket science. Loading 9 additional instances when you expect 20 is not smart.
YMMV - Your mileage may vary. Try it both ways and see.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| paintpot |
| Posted: Tue Feb 28, 2012 9:42 am Post subject: |
|
|
Centurion
Joined: 19 Sep 2005
Posts: 112
Location: UK
|
[quote="lancelotlinc"]
| deepnair wrote: |
| Quote: |
| Best TPS is Approach 2. |
YMMV - Your mileage may vary. Try it both ways and see. |
Yep, absolutely. And don't forget the answer may change depending on what else you use this box for / how you have it configured / how you tune it |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Feb 28, 2012 9:49 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| paintpot wrote: |
Transactionality? Is it valid to complete in multiple units of work?
Time to complete each 'chunk'. If any of the parts take a long time to complete, you will hold less resources / have better throughput if you split it up with MQ. You can then balance the work / latency by having multiple copies at the right point.
Partial reuse? Will other applications want to make use of the 'bits' you suggest in diffferent ways?
As usual - it depends... |
What happens to a given transaction if C fails? Do you continue with D? Do you need to reverse (backout) A & B?
Questions, questions...
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Feb 28, 2012 9:49 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Another consideration is rate of change.
If function C is undergoing heavy development and will be expected to change on a frequent basis over the next several months, you may wish to isolate that function to it's own flow to minimize the amount of regression testing you need to do, and to minimize the change management in general. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Feb 28, 2012 9:52 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| lancelotlinc wrote: |
| Adding MQ to the latency ncreases your latency around 13 msec per queue (you suggest 3 queues, total increase would be 39 msecs). |
YMMV. Using message persistence on a poorly tuned queue manager could be higher than this, non-persistent messages may go significantly faster.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| deepnair |
| Posted: Tue Feb 28, 2012 12:32 pm Post subject: |
|
|
Apprentice
Joined: 07 Feb 2012
Posts: 35
|
ok, so it seems the answer will be it depends 
A general conclusion may be
Individual flows approach
-- Good when non-persistant queues are being used (hops relatively consume less time)
-- Good when rate of change is high (regression eases)
-- Good if the functions are heavy or time consuming (thus the thread latency will be minimum)
-- Good if all these functions are for a framework flow, as the overall load on framework can vary, thus the overall allocated threads can balance the tps needs
Subflow approach
-- Good for better performance (subflows bound statically, no hops)
-- If the functions are simple, subflows dont take too much time in processing and they dont hold the thread for a long time
-- MQ latency can be avoided.
Also exception handling will not be a challenge in either of these approaches as it can be defined as a separate flow and deployed in a different EG altogether thus decoupling the exceptions from the main logic.
Thanks,
Deep Nair |
|
| Back to top |
|
|
| inMo |
| Posted: Thu Mar 01, 2012 11:53 am Post subject: |
|
|
Master
Joined: 27 Jun 2009
Posts: 216
Location: NY
|
| Quote: |
Subflow approach
-- Good for better performance (subflows bound statically, no hops)
|
Well, no, not always. It depends. In example, if the subflows are all resource hogs, linking them up to be a resource monster may actually slow things down. Adding additional instances to the single flow to further input will open a number of threads all competing for the resources.
In contrast, accepting the rather miniscule MQ latency in exchange for 'options' may actually increase performance. Perhaps Flow B is such a hog that you will be better served by isolating it in a dedicated EG. |
|
| Back to top |
|
|
|
|
