Posted: Tue May 12, 2009 4:12 pm Post subject: Finding my performance constraint
Chevalier
Joined: 05 Nov 2005 Posts: 401
Broker 6.1
AIX 5.3
4GB RAM
CPUs allocated on demand
Sorry for the very general nature of this question, but I'm just looking for the most likely causes that we should investigate.
We have a flow which receives a message and makes a 5 external calls before returning. The calls have dependencies so have be executed one after the other. The processing is CPU bound. Under light load everything is fine, the CPUs 'light up' to a reasonably high percentage. However when we ramp up the concurrency beyond a certain point the %CPU usage goes down !! , there is some constraint to thruput here that we can't figure out. The performance of the external calls remains high.
We increased additional instances of the flow and initially saw roughly linear scaling. However once we increased additonal instances of the flow to cope with the maximum arrival rate of messages that we will need (I think we set it around 50) we see a dramatic fall off in thruput, i.e the end-2-end performance degrades (a lot).
Is it likely that resources required by Broker to support the additonal instances are running short ??
Joined: 15 Jun 2005 Posts: 163 Location: Massachusetts
50 instances (per flow) and each one operating 5 threads (for responses) puts this right at the operation system limit for the no.of practical concurrent threads per EG. IBM Recommended working limits are 256 threads per process for Windows and UNIX and 230 TCBs per Address Space on z/OS
The best way to mitigate this would be to spread the instances of the flow to different execution groups. Hope this helps!
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum