| Author |
Message
|
| Veera B |
 Posted: Thu Mar 28, 2013 8:04 pm Post subject: Dif b/w increasing flow instance in flow i/p nodes and BAR f |
 |
|
Voyager
Joined: 16 Jan 2013
Posts: 76
|
Hi all,
Need to increase the troughput of a purticular flow, for which even after increasing the additional instances property in bar file to 256 is not enough.
so, i was thinking if i increase the additional instances property in MQInputnode also to 256, will the number of threads become 512 ?
256 from BAR file value and 256 from MQinput node value .. is it correct or is it still going to be 256 threads of the flow ?
Pls advise ... |
|
| Back to top |
|
 |
| Esa |
| Posted: Thu Mar 28, 2013 11:25 pm Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
No, because then all instances would be allocated from the MQInput instance pool only. Unless you have another MQInput node in the flow.
But I'm not sure if the max number of a flow in an execution group can exeed 256 even if you had several input nodes with dedicated pools.
The question has not crossed my mind. Why? Because normally there is a limit obove which adding instances does not improve performance,
and that limit is way under 255, often 10 - 20. Above that there will be shortage of some resources, memory for example, and adding instances would just worsen the situation.
Have you tested the performance of the flow with less additional instances? Maybe the optimal number of instances is something like 30 or 50?
Have you analyzed the flow? HAve you run statistics? What is the average execution time? What is the average execution time with no additional instances?
Maybe there is something wrong with the design?
How many input messages do you get? Thousands per second?
You can get 256 more instances by deploying the flow in an additional execution group. That will help - if you do have enough resources, like memory. In which case an additional processor for the new EG might really boost your flow. |
|
| Back to top |
|
|
| adubya |
| Posted: Fri Mar 29, 2013 3:11 am Post subject: |
|
|
Partisan
Joined: 25 Aug 2011
Posts: 377
Location: GU12, UK
|
| Esa makes excellent points. Before I started throwing more instances at a problem I'd look into the performance and design/structure of a flow. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Fri Mar 29, 2013 4:13 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| adubya wrote: |
| Esa makes excellent points. Before I started throwing more instances at a problem I'd look into the performance and design/structure of a flow. |
Round wheels roll better, faster, with less effort. Egg shaped wheels take more energy to motivate and don't roll very far. Adding more egg-shaped wheels won't help you much.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Veera B |
| Posted: Sun Mar 31, 2013 8:32 pm Post subject: |
|
|
Voyager
Joined: 16 Jan 2013
Posts: 76
|
Ok .. here's the whole scenario .. we have a flow which uses JCN to put and get messages from a MQ adapter and few other places JCN's have been used .. its a bit complex flow ...
The input msg's are sent in batches of 400-500.. first initial 100-150 msg's there are no problems .. the response times are within 5-10 secs. but from here response times start getting more and more .. it reaches beyond 30 secs per msg and the last few msg's get timed out.
flow has 256 threads and if i add another 256 into different EG .. the timeouts are solved [ still resp time not great 20 - 25 secs for few ], but that is not acceptable in prod and flow design can't be changed [ only minimal cofig changes ].
also the JVMheapsize for EG:- min - 33mb, max is mentioned -1(assuming infinite).
i accept flow design is not great, but i think this has more to do with memory utilization and flow threads not closing soon enough .. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Mar 31, 2013 10:41 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Veera B wrote: |
Ok .. here's the whole scenario .. we have a flow which uses JCN to put and get messages from a MQ adapter and few other places JCN's have been used .. its a bit complex flow ...
The input msg's are sent in batches of 400-500.. first initial 100-150 msg's there are no problems .. the response times are within 5-10 secs. but from here response times start getting more and more .. it reaches beyond 30 secs per msg and the last few msg's get timed out.
flow has 256 threads and if i add another 256 into different EG .. the timeouts are solved [ still resp time not great 20 - 25 secs for few ], but that is not acceptable in prod and flow design can't be changed [ only minimal cofig changes ].
also the JVMheapsize for EG:- min - 33mb, max is mentioned -1(assuming infinite).
i accept flow design is not great, but i think this has more to do with memory utilization and flow threads not closing soon enough .. |
Contrary to your assumption, I would suspect poor flow design first... It's what you get when a java developer is tasked with writing a JCN without the proper training... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| adubya |
| Posted: Sun Mar 31, 2013 11:41 pm Post subject: |
|
|
Partisan
Joined: 25 Aug 2011
Posts: 377
Location: GU12, UK
|
| Veera B wrote: |
Ok .. here's the whole scenario .. we have a flow which uses JCN to put and get messages from a MQ adapter and few other places JCN's have been used .. its a bit complex flow ...
|
MQ Adapter ? Do you mean the flow doesn't use native broker MQ Input/Output nodes ? |
|
| Back to top |
|
|
| Veera B |
| Posted: Mon Apr 01, 2013 1:33 am Post subject: |
|
|
Voyager
Joined: 16 Jan 2013
Posts: 76
|
It uses broker MQ Input/Output nodes only, but to connect to backend application, it is using MQ adapter. i mean message exchange to backend is happening trough MQ adapter.
and MQput and MQget to this MQ adapter is being done from a java compute node, which i believe is causing the delay in response times ... |
|
| Back to top |
|
|
| adubya |
| Posted: Mon Apr 01, 2013 2:26 am Post subject: |
|
|
Partisan
Joined: 25 Aug 2011
Posts: 377
Location: GU12, UK
|
Well I'm confused 
The flow only uses MQInput/Output nodes but a JCN uses MQPut/Get to communicate to the adapter ? You'll have to clarify that for me.
Plus, as you mentioned MQPut/Get, is the flow taking part in a synchronous exchange with the adapter as part of its message processing ? You'll have to explain the high-level structure of your flow. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Apr 01, 2013 4:37 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Veera B wrote: |
| It uses broker MQ Input/Output nodes only, but to connect to backend application, it is using MQ adapter. i mean message exchange to backend is happening trough MQ adapter. |
What adapter? Some 3rd party thing? Do you really mean an adapter as in a piece of software, or just an MQ interface to the backend software?
| Veera B wrote: |
| and MQput and MQget to this MQ adapter is being done from a java compute node, which i believe is causing the delay in response times ... |
As written, what you've described is a hideously bad idea. Doing any kind of MQ message activity directly from code and not from the broker's supplied MQ nodes is a one way path to pain and misery. At best it causes the sort of problem you're seeing, at medium it causes the MQ transactionality to not work properly (or at all) and at worst it will bring the broker and/or execution group over.
As my most worthy associate pointed out earlier, this is a poor flow design and exactly the sort of thing you get when you give a Java developer a broker task without proper training and guidance.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Apr 01, 2013 12:05 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| the response times are within 5-10 secs. but from here response times start getting more and more .. it reaches beyond 30 secs per msg and the last few msg's get timed out. |
There is only one solution to this problem: carry out some performance tests to find out why this slowdown is happening.
All other solutions are doomed from the start because the message flow is complex and badly-designed. Throwing more threads at this flow is like fitting a v8 engine to a small family car when the correct solution is to repair the sticking brakes. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Apr 01, 2013 12:33 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Or the performance bottleneck is in that back end app and the WMB Flow, bad design notwithstanding, simply overwhelms it with queued requests.
Increasing the throughput of the WMB flow could have the contradictory effect of causing it to time out on even a larger percentage of requests it sends if all that does is cause that back end's queue to get deeper faster.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Veera B |
| Posted: Mon Apr 01, 2013 9:44 pm Post subject: |
|
|
Voyager
Joined: 16 Jan 2013
Posts: 76
|
| Vitor wrote: |
| Veera B wrote: |
| It uses broker MQ Input/Output nodes only, but to connect to backend application, it is using MQ adapter. i mean message exchange to backend is happening trough MQ adapter. |
What adapter? Some 3rd party thing? Do you really mean an adapter as in a piece of software, or just an MQ interface to the backend software?
|
its an MQ Adapter to connect and send/recieve messages from backend host. Also the connecting part is done in JCN |
|
| Back to top |
|
|
| Esa |
| Posted: Mon Apr 01, 2013 10:16 pm Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| There certainly is something wrong with the design, but you are also giving the flow too many threads. They exhaust the system. Try with different numbers of additional instances: 10, 20, 30, 40 etc. to find the optimal number. When the performance starts to decrease you have hit the limit. |
|
| Back to top |
|
|
| Veera B |
| Posted: Tue Apr 02, 2013 7:07 pm Post subject: |
|
|
Voyager
Joined: 16 Jan 2013
Posts: 76
|
| Esa wrote: |
| There certainly is something wrong with the design, but you are also giving the flow too many threads. They exhaust the system. Try with different numbers of additional instances: 10, 20, 30, 40 etc. to find the optimal number. When the performance starts to decrease you have hit the limit. |
Yep, its true. more instances causes resource crunch.
Also for the same broker there are many other Eg's which are running flows with 256 instances .. so when many ppl are testing, the system struggles to allocate memory.
I tested 500+ msgs at a time, when no one is testing, with only 15 instances.
The response times decreased 10 times when compared to 256 instances. |
|
| Back to top |
|
|
|
|
