| Author |
Message
|
| petermqiis |
 Posted: Wed Jun 17, 2009 2:47 am Post subject: MQ and IIS integration |
 |
|
Novice
Joined: 17 Jun 2009
Posts: 10
|
We have a legacy web app that runs on IIS accepting http requests. We would now like to respond to MQ messages and respond back over MQ. One option is to write a monitor to poll the queue, make http connections and post messages back on the queue, however, this doesn't seem robust since if the monitor goes down there is nothing to pick it up again and it doesn't scale particularly well.
What would be the usual solution to this problem?
Thanks in advance |
|
| Back to top |
|
 |
| Vitor |
| Posted: Wed Jun 17, 2009 2:56 am Post subject: Re: MQ and IIS integration |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| petermqiis wrote: |
| What would be the usual solution to this problem? |
Have the app monitoring the queue triggered by a message. Because the monitor is only running when there's a message to process, this eliminates the problem of it crashing during a period of inactivity, and is more efficient than polling.
Similar techniques can be used to scale the solution by spawing additional instances of the app to process high message loads (if IIS can handle that, I know nothing of it).
If you want total throughput, reliability and scalability, buy WMB and get that to make the http requests! 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| petermqiis |
| Posted: Wed Jun 17, 2009 4:01 am Post subject: |
|
|
Novice
Joined: 17 Jun 2009
Posts: 10
|
I've run the example code for a .net trigger but it is still invoked from a command line process namely runmqdnm. If my code throws an exception this process dies and there is no monitoring left.
Our app since it is running in IIS has no worker processes it only reponds to http requests so it can't directlt interact with the queue.
In what environment should the mq-http converter be running. So long as it's just a windows process it's vulnerable. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Jun 17, 2009 4:07 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Look at the MA7K Support Pack, which allows you to run the MQ Client trigger monitor as a Windows Service. Works very well - been using it for years for multiple apps.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jun 17, 2009 4:08 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| petermqiis wrote: |
| it is still invoked from a command line process namely runmqdnm. |
It says here:
| Quote: |
| runmqdnm can be run from the command line, or as a triggered application. |
I was suggesting the latter
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| petermqiis |
| Posted: Wed Jun 17, 2009 4:20 am Post subject: |
|
|
Novice
Joined: 17 Jun 2009
Posts: 10
|
Great that looks promising
How is that configured? The documentation seems to only list the command line parameters.
btw - the solution needs to work at around 50-100 messages per second. So I'm not sure if starting a new application each time will be fast enough. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jun 17, 2009 4:25 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| petermqiis wrote: |
| How is that configured? The documentation seems to only list the command line parameters. |
Triggering is described in the Application Programming Guide & the System Admin manual.
| petermqiis wrote: |
| btw - the solution needs to work at around 50-100 messages per second. So I'm not sure if starting a new application each time will be fast enough. |
I'm not sure starting a new application each time is best design if you're processing 50 - 100 messages a hour! Your application should start on first message then keep running until it's cleared the queue (which is clearly never if more messages turn up). Use a separate mechanism to detect if the arrival rate of messages is running ahead of the application's speed of processing and respond accordingly.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| petermqiis |
| Posted: Wed Jun 17, 2009 5:01 am Post subject: |
|
|
Novice
Joined: 17 Jun 2009
Posts: 10
|
That's the root of the problem what is a good environment for monitoring that process? I was hoping some MQ component would be fault resilient and able to trigger code in a lightweight way just like IIS or Apache does when a new http request comes along. Except in this case it's an MQ message. If IIS had worker processes it could be written in that environment.
Should I be looking at something like host integration services?
The solution doesn't necessarily need to be low cost. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jun 17, 2009 5:10 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| petermqiis wrote: |
| The solution doesn't necessarily need to be low cost. |
WMB then.
Seriously, monitor what? The trigger monitor (as my most worthy associate correctly says) is reliable and can run as a service. There's no need to monitor the starting of the application, only the throughput. And only then if it's an issue.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| petermqiis |
| Posted: Wed Jun 17, 2009 5:28 am Post subject: |
|
|
Novice
Joined: 17 Jun 2009
Posts: 10
|
Except that it passes control to the user code in my understanding. So in my example I throw an exception and the whole process dies, monitor and all. Unless the monitor can become a host environment it won't serve our purposes. This system will be processing upto 100 messages per second 24/7 with expected uptime of 99.7%. IIS can give us these guarantees and MQ is very robust but having a lone windows process in the middle is an unacceptable single point of failure.
Imagine if while processing a web page an exception brought the whole of IIS or Apache down! |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jun 17, 2009 5:30 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| petermqiis wrote: |
| So in my example I throw an exception and the whole process dies, monitor and all. |
The trigger monitor runs as a separate process to the triggered application.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| petermqiis |
| Posted: Wed Jun 17, 2009 6:10 am Post subject: |
|
|
Novice
Joined: 17 Jun 2009
Posts: 10
|
I'll take a look but I'm scpetical that having upto 100 triggered application instances starting up per second each creating it's own http connection will be a good idea.
I'm still really betting on having some piece of infrastructure to manage a long running process that can pool http connections but have new threads brought online whenever necessary in case of failure or high load.
I've been lead to believe that MQ is industrial strength so a weak link will be rejected by our client if we can't assure them that this is a tried and tested solution.
Thank for all your help. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jun 17, 2009 6:16 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| petermqiis wrote: |
| I'll take a look but I'm scpetical that having upto 100 triggered application instances starting up per second each creating it's own http connection will be a good idea. |
When did I suggest that? 
| petermqiis wrote: |
| I'm still really betting on having some piece of infrastructure to manage a long running process that can pool http connections but have new threads brought online whenever necessary in case of failure or high load. |
Well this is your design decision to make.
| petermqiis wrote: |
| I've been lead to believe that MQ is industrial strength so a weak link will be rejected by our client if we can't assure them that this is a tried and tested solution. |
It's extremely industrial and I would not consider a message throughput of 100 messages per second, with the uptime you've indicated, anything unusual. Subject to hardware resource obviously.
Certainly the scenario you're describing is unremarkable.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| petermqiis |
| Posted: Wed Jun 17, 2009 6:24 am Post subject: |
|
|
Novice
Joined: 17 Jun 2009
Posts: 10
|
I'm not sure I understand. It seems to me I have 2 options:
The commandline monitor, single instance of my code forwarding messages to http. I can pool a limited number of http connections and marshal results back. If it my code throws an exception (as I have tested) the whole monitor dies with no failover. Manual intervention is required.
The service monitor starting a new spplication instanceper message. No pooling of http connections available since each instance is an isolated process. This is fault tolerant but scalable.
So the first scales but is no robust. The second doesn't scale but is robust. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jun 17, 2009 6:40 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| petermqiis wrote: |
| IThe commandline monitor, single instance of my code forwarding messages to http. I can pool a limited number of http connections and marshal results back. If it my code throws an exception (as I have tested) the whole monitor dies with no failover. Manual intervention is required. |
Yes, this sucks.
| petermqiis wrote: |
| The service monitor starting a new spplication instanceper. |
Yes, this is good.
| petermqiis wrote: |
| So the first scales but is no robust. The second doesn't scale but is robust. |
You seem to be contradicting yourself a little here. But the second solution scales as well as the first, unless I'm missing something significant in your set-up (which is not impossible), and as you say is fault tolerant.
I think the point here is why can you pool http connections if the application is run from the command line, but not if it's triggered? This is the point I find hard to understand.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
