| Author |
Message
|
| moonwalker |
 Posted: Thu Feb 12, 2015 2:03 pm Post subject: Stop Broker queue manager when broker stops. |
 |
|
Apprentice
Joined: 02 Jun 2009
Posts: 42
|
Hello All,
For one of our clients we are setting up an environment where we have a load balancing setup which includes a load balancing QM and two broker QMs all in a cluster. The upstream tries to put message to the load balancing QM which distributes messages to broker queue managers in a round robin fashion. Since both the broker are running exactly same set of flows we are achieving a load balance here. At the same time we have built a way of high availability as well. No troubles or problems with this approach.
Now, recently, we hit road block ever since the message sizes have increased resulting in broker getting hanged. Now the actual problem statement is, incase if a broker hangs or stops since we frequently transact huge sized messages, the corresponding QM still seeks messages from the load balancing QM until the max depth of the queue is reached. So how do we curtail the QM from receiving message when its associated broker has stopped for any reason?
We are completely fine with the broker going down as we have multiple servers with the same setup. But we want the associated QM also to go down when broker stops responding.
Ever since then I am exploring ways to tie the broker and corresponding broker QM process so both go down in the event of either getting stopped.
I know there might be ways of handling large sized messages but I am interested towards stopping the broker QM when broker stops.
One solution that I thought of was to have an external shell or java based script which monitors broker and its associated QM's health. If broker stops responding then issue a command to stop the associated broker QM. But I have this option as a last resort.
So please let me know if there are any built in features which supports are requirement? |
|
| Back to top |
|
 |
| PeterPotkay |
| Posted: Thu Feb 12, 2015 5:21 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Google for amqsclm
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| moonwalker |
| Posted: Thu Feb 12, 2015 9:36 pm Post subject: |
|
|
Apprentice
Joined: 02 Jun 2009
Posts: 42
|
| Seems a tailor made solution for my requirement. thanks for responding. |
|
| Back to top |
|
|
| moonwalker |
| Posted: Fri Feb 27, 2015 6:44 am Post subject: |
|
|
Apprentice
Joined: 02 Jun 2009
Posts: 42
|
Amqsclm does a descent job in moving traffic across clustered queue managers. So if one broker goes down unexpectedly, then the tool moves messages on to other broker queue manager part of the cluster.
After this was solutioned a week ago, everything looked good until one of the brokers went down yesterday. The solution was robust.
But the tool doesn't send notifications or reports that one of the broker is down? I know its a sample tool but need to find an alternative of seeking notification if the broker has gone down.
The reason we need to be notified is to correct or resolve the issue on the stopped broker for it to be restarted.
Tivoli monitoring is an option but client is not willing to license it. So we r indeed out of options.
Last edited by moonwalker on Fri Feb 27, 2015 6:47 am; edited 1 time in total |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Feb 27, 2015 6:47 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
There are lots of relatively easy ways to determine that a process has actually stopped, and then create something that resembles an 'alert'.
I suspect your customer already has some kind of monitoring system in place already, that could be adapted to monitor this stuff and create an event to respond. |
|
| Back to top |
|
|
| moonwalker |
| Posted: Fri Feb 27, 2015 6:50 am Post subject: |
|
|
Apprentice
Joined: 02 Jun 2009
Posts: 42
|
Monitoring at the process level? You mean a OS level monitor?
It would be helpful if u may elaborate please. Thanks |
|
| Back to top |
|
|
| adubya |
| Posted: Fri Feb 27, 2015 7:42 am Post subject: |
|
|
Partisan
Joined: 25 Aug 2011
Posts: 377
Location: GU12, UK
|
| We monitor processes and queue depths. If a required process isn't running then we alert. If a certain queue exceeds a depth threshold then we alert. |
|
| Back to top |
|
|
| moonwalker |
| Posted: Fri Feb 27, 2015 1:46 pm Post subject: |
|
|
Apprentice
Joined: 02 Jun 2009
Posts: 42
|
| Exactly, there are whole lot of tools which do monitoring and alerting separately. So is there anything which does both together? Basically a single tool which might monitor the health of the broker and also alert when it goes down. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Feb 27, 2015 1:51 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| moonwalker wrote: |
| Basically a single tool which might monitor the health of the broker and also alert when it goes down. |
| moonwalker wrote: |
| Tivoli monitoring is an option but client is not willing to license it |
|
|
| Back to top |
|
|
| smdavies99 |
| Posted: Fri Feb 27, 2015 10:43 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| moonwalker wrote: |
| Tivoli monitoring is an option but client is not willing to license it |
There are other options that (IMHO) are a lot better than Tivoli. To be honest I've given up on Tivoli and it is not for want of trying.
Getting the Agents to work reliably has been its Achillies heel as far as I'm concenred.
I'm about to start a POC with another vendor's solution. It promises to be a lot more robust because it isn't agentless.
For small sites tools like Tivioli, QPasa (or whatever it is called now) are very expensive simply because they are Enterprise solutions.
If the customer is truly unwilling to stump up for a decent monitoring solution (not exactly uncommon) and wants to roll their own then having done that around 2005 I'd be on the lookout for another position. If you see my signature you should get my opinion.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
|
|
