| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Automatically suspending broker activity. |
« View previous topic :: View next topic » |
| Author |
Message
|
| jgooch |
 Posted: Tue Jan 13, 2004 2:12 am Post subject: Automatically suspending broker activity. |
 |
|
Acolyte
Joined: 29 May 2002
Posts: 63
Location: UK
|
Hi,
We had a problem recently where our network went down. When it came back MQ delivered all the in-flight messages to WMQI. However, the network failure had killed the brokers' ODBC connection to our key map and message management database. Hence thousands of messages ended up in failure queues and it took a lot of effort to recover them all.
Is it possible to suspend broker activity automatically when the first message detects such a problem? This will give our support team the necessary time to bounce the broker and re-establish the connection.
How do other people manage this issue? Should we implement a "heartbeat" flow that can then trigger the appropriate scripts? Is there a way to force the broker to re-establish its connection without having to restart it?
Obviously, we could use the MQGetDisable support pack but we have 300+ flows and cannot afford the retesting effort involved in changing them all.
We're using WMQIv2.1 CSD04, MQv5.3 on Solaris v5.8.
Thanks,
J. |
|
| Back to top |
|
 |
| Nick Lethbridge |
| Posted: Tue Jan 13, 2004 5:40 am Post subject: |
|
|
Voyager
Joined: 13 Aug 2001
Posts: 88
Location: Santander, UK
|
If you use a third party monitoring product (e.g. QPasa, BMC Patrol, Candle) you could probably configure an application or script to be run if (say) 5 messages or more are written to your failure queue.
You could then create a script to telnet to the broker server, logon as the WMQI user and issue a mqsistop against the broker ?
Regards,
Nick. |
|
| Back to top |
|
|
| fschofer |
| Posted: Tue Jan 13, 2004 6:40 am Post subject: |
|
|
Knight
Joined: 02 Jul 2001
Posts: 524
Location: Mainz, Germany
|
Hi,
with the Candle Monitoring Node you can create "events" when a message goes through the monitoring node.
You can use these events to stop a flow
I use this in my failure handling to stop flows when a critical error occurs.
But the message flow stop will take some time so that some messages still will be put in your error queue.
I also had once some problems with ODBC database connections,
but usually the broker reconnected after the first failed database access.
So i catch database errors and throw an exception to cause a flow rollback and give it a second try.
You can limit the number of retries by using the MQ backout mechanism and setting the backout counter. |
|
| Back to top |
|
|
| kirani |
| Posted: Tue Jan 13, 2004 10:09 am Post subject: |
|
|
Jedi Knight
Joined: 05 Sep 2001
Posts: 3779
Location: Torrance, CA, USA
|
This is more of architecture problem than WMQI. The broker will cache the DB connection when it first tries to connect to an external DB. For all subsequent messages, it will reuse the same connection. During the processing of the message if it finds a dead connection, the broker will throw an exception and it will drop the dead connection from the cache. It's up to your message flow to decide what to do with the failed message. Next time, when it tries to access the same DB, it will create a new connection.
In our message flows we are handling this error by setting the backout Threshold property to 1 on the input queue. This will allow our messages to process 2 times in case of failure conditions, next time the message is processed successfully in this scenario.
You can design your error handler to disable the input queue, which will eventually stop your message flow processing. We are using MQDisable plug-in node in our error handler sub-flows to do this job.
_________________
Kiran
IBM Cert. Solution Designer & System Administrator - WBIMB V5
IBM Cert. Solutions Expert - WMQI
IBM Cert. Specialist - WMQI, MQSeries
IBM Cert. Developer - MQSeries
|
|
| Back to top |
|
|
| RatherBeGolfing |
| Posted: Tue Jan 13, 2004 10:39 am Post subject: |
|
|
Centurion
Joined: 12 Nov 2002
Posts: 118
Location: Syracuse, NY, USA
|
Could you also plumb another MQOutput node on the fail terminal and send a message to a triggered Q which would launch a .bat file. The .bat file could contain a runmqsc command to disable the GET attribute of 1 or more Qs, effectively stopping the flow(s) from processing all of the messages which are going to fail due to the ODBC problem. You may have to use the FORCE option.
You could also set up your monitoring software to watch the same triggered Q for a depth of 1. Or, if you don't run monitoring software, you could invoke your own sendmail routines to page oncall people, etc.
_________________
Cheers,
Larry
MQ Certifiable |
|
| Back to top |
|
|
| jgooch |
| Posted: Wed Jan 14, 2004 1:11 am Post subject: |
|
|
Acolyte
Joined: 29 May 2002
Posts: 63
Location: UK
|
Thanks, guys. Good replies so far!
I was 97.3% certain (  ) that QPasa, Patrol or Omegamon could do this. I've recommended acquiring such a tool several times now, but they go running for the hills when you mention the likely cost. In the meantime, they want real-time pro-actively monitored and recoverable systems... 
I'll try the back-out threshold change and let you know how I get on. To be honest, I'm not sure I've ever seen the broker re-establish a broken ODBC connection itself. Methinks I'll do some tests with our DBAs to be sure.
J. |
|
| Back to top |
|
|
| kirani |
| Posted: Wed Jan 14, 2004 11:51 pm Post subject: |
|
|
Jedi Knight
Joined: 05 Sep 2001
Posts: 3779
Location: Torrance, CA, USA
|
I personally didn't see any value into Candle WMQI monitoring product. As compared to the product cost it's not doing very much  One can write couple of system programs to achieve similar functionality. We have custom written programs for monitoring WMQI and MQ components in our environment.
_________________
Kiran
IBM Cert. Solution Designer & System Administrator - WBIMB V5
IBM Cert. Solutions Expert - WMQI
IBM Cert. Specialist - WMQI, MQSeries
IBM Cert. Developer - MQSeries
|
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
