| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| http request node load |
« View previous topic :: View next topic » |
| Author |
Message
|
| matuwe |
 Posted: Wed Nov 09, 2011 4:33 am Post subject: http request node load |
 |
|
Master
Joined: 05 Dec 2007
Posts: 296
|
Hi
I have three flows,
1. MQ in >> compute >> mqout
2. MQ in >> http request >> mqout
3. MQ in >> compute >> mqout
My issue is around flow 2. So if I receive 5000 messages all at the same time, them flow two processing gets slow, mainly because I am relying on http Request to respond before I can process the next messge.
I have tried increasing number of instances, but all it means is I can only handle 5 or n no of http requests.
Is there anyway I can auto increase no of instances for a flow, or some magic to handle the load, design change?? Dont know what to do
Help
   |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Wed Nov 09, 2011 4:36 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Yes, HTTP is a synchronous protocol.
Yes, the way to address this is to increase the number of additional instances on the middle flow until you have an "n" that matches your need.
EDIT: additional instances are allocated and used on demand. So if you set N = 100, you will not always have 100 instances running. You'll only use that full number if you have 100 simultaneous messages on the input queue and all of them take longer to finish than the time needed to get the next message.
And then performance tune the http service you are making the request of. You have clearly identified it as a performance bottleneck. |
|
| Back to top |
|
|
| matuwe |
| Posted: Wed Nov 09, 2011 7:28 am Post subject: |
|
|
Master
Joined: 05 Dec 2007
Posts: 296
|
 oowww that is good to know. I was always under the impression all 100 instances will be running at the same time.
Will it be bad practice if I increment at lot of my flows where there are bottleneck? How does it impact the CPU usage if at any point all 100 instances are used? Will I maybe run into an OutOfmemory?
Thanks a million for the information , much appreciated,  |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Nov 09, 2011 7:33 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| matuwe wrote: |
| How does it impact the CPU usage if at any point all 100 instances are used? |
100 instances use a lot more CPU than 1 instance.
| matuwe wrote: |
| Will I maybe run into an OutOfmemory? |
Maybe. 100 instances use a lot more memory than 1 instance. Will you run out? Depends how much memory you have.
I'd stick with the suggestion of my most worthy associate and tune whatever's on the http request so it's not a bottleneck & you can achieve greater throughput with fewer instances.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Wed Nov 09, 2011 7:35 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| matuwe wrote: |
Will it be bad practice if I increment at lot of my flows where there are bottleneck? How does it impact the CPU usage if at any point all 100 instances are used? Will I maybe run into an OutOfmemory?
|
Test, Test, Test and Test again.
Honestly, there is no set formula here. You just have to take your time, be methodical and eventually, you will hit the sweet spot.
Remember that throwing resources at a bottleneck only serves to move the bottleneck. Granted, you may get some performance improvement but you will then have to look elsewhere for the next increment. Rinse, repeat and pray until you get the throughput you need.
For example,
The original setup gave us 16 messages/sec. Eventually, we got to 60/sec. In live oiperation it was doing 75/sec day in/day out. It did take time to get to the bottom of the various bottlnecks. This included SQL/Server performance, SAN performance and even how the remote SQL Server ODBC Connections were established (moved to full dup, no negociation) as well as making code changes to our flows to remove DB locking.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
