| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Achieving zero down time web services |
« View previous topic :: View next topic » |
| Author |
Message
|
| Vitor |
 Posted: Wed Aug 13, 2014 4:56 am Post subject: Achieving zero down time web services |
 |
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
IIBv9.0.0.2 on a virtual AIX
Our scenario has multiple applications, typically one application with 1-n flows to an execution group, with execution group listeners and an F5 load balancer in front of the broker.
We have a business need to detect if an individual flow has become
unresponsive and switch traffic to a different server, where it's acceptable to switch all traffic not just the operation hosted by the failing flow. Currently the inbuilt ability of the F5 is unsuitable, as this acts on the port and switches if the port becomes unresponsive; so long as one flow is
running the EG listener remains active and the port appears live to the
F5. Adding one flow per EG resolves this but at a huge resource and
administrative cost of many, many EGs. Using workload management
profiles to detect an unresponsive flow is another option, but this only allows a restart of the entire EG. Of course this brings down the EG listener and hence signals the F5, but we lose throughput while the EG restarts and in the event the flow is unresponsive due to an external resource like a database being hung up, the EG could churn in an unwanted manner.
The business are unwilling to accept that the EG will simply restart a failed flow and thus traffic does not need to be re-routed for the few seconds a restart will take, and fear that a flow may be stopped administratively and require traffic to be diverted.
Does anyone have any good ideas for this which don't involve telling the business not to be ridiculous?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
 |
| zpat |
| Posted: Wed Aug 13, 2014 5:07 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
One of the many problems with using web services instead of MQ for integration is the nature of stateless push connections.
The EG listener will accept connections even when all message flows are busy or hung - the length of this internal queue can be tuned. Having a longer queue can offer some buffering, but may delay the discovery by the F5 that the flows aren't working.
However in your case - reducing the size of the buffer may help expose the fault sooner to the F5. Look at these options for the http(s) connector - maxThreads and acceptCount
Each incoming request consumes a tomcat thread to process and maxThreads controls the size of the Tomcat thread pool. The accept count parameter controls how many connection requests are accepted and queued when all tomcat threads are in use.
http://www-01.ibm.com/support/docview.wss?uid=swg21504029
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Aug 13, 2014 5:21 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Also, this is the kind of thing that WAS tends to solve using the HTTP server tier as a load-balancer...
And it's easy enough to export the HTTP/SOAP config of Broker v8 and later into either a WAS plugin config or a plain Apache mod-proxy config.
Food for thought. But even then, you'll probably need to do the tuning that zpat mentions. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Aug 13, 2014 5:25 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
I have seen sites running a single admin flow in each e.g.
This was run in a JCN using the broker API. The F5 queries the flow at regular interval and the flow returns some XML... like e.g. name, flow name, status, etc...
Caveat: make sure you do not create a memory leak using the broker API.
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Aug 13, 2014 5:29 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| fjb_saper wrote: |
I have seen sites running a single admin flow in each e.g.
This was run in a JCN using the broker API. The F5 queries the flow at regular interval and the flow returns some XML... like e.g. name, flow name, status, etc...
Caveat: make sure you do not create a memory leak using the broker API.
Have fun |
Again, that merely says that the EG is up and running, not that a given flow within that EG is active.
  |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Aug 13, 2014 5:34 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| mqjeff wrote: |
| fjb_saper wrote: |
I have seen sites running a single admin flow in each e.g.
This was run in a JCN using the broker API. The F5 queries the flow at regular interval and the flow returns some XML... like e.g. name, flow name, status, etc...
Caveat: make sure you do not create a memory leak using the broker API.
Have fun |
Again, that merely says that the EG is up and running, not that a given flow within that EG is active.
|
I think you need more coffee... F5 will act on the return XML and know whether a flow is running or stopped as this gets queried with the broker API and is part of the return XML...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Aug 13, 2014 5:37 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| fjb_saper wrote: |
| mqjeff wrote: |
| fjb_saper wrote: |
I have seen sites running a single admin flow in each e.g.
This was run in a JCN using the broker API. The F5 queries the flow at regular interval and the flow returns some XML... like e.g. name, flow name, status, etc...
Caveat: make sure you do not create a memory leak using the broker API.
Have fun |
Again, that merely says that the EG is up and running, not that a given flow within that EG is active.
|
I think you need more coffee... F5 will act on the return XML and know whether a flow is running or stopped as this gets queried with the broker API and is part of the return XML... |
Yes, I agree, if all you have is an F5, you can make horses carry boats.
But given that Apache is free to use and install... |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
