| Author |
Message
|
| Vitor |
 Posted: Mon Mar 15, 2010 8:49 am Post subject: Overhead of performance events |
 |
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Does anyone have a link / redbook / etc that describes the overhead imposed on a queue manager generating performance events?
We have v6 Windows queue manager on an admitedly underpowered server and are looking to implement a Queue Service Interval event on a single queue. I'm getting push back from the client's MQ admin who's claiming that the server "won't support that additional load" and "any halfway decent WMQ admin knows better than to use events".
This position is a little light on evidence apart from anecdotal evidence that she was at a tech conference once and everyone was agreeing that Events Kill Queue Managers, but she has the advantage of position and credibility. Before I shrug and go, "whatever" I was wondering if there was any print I could throw...
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
 |
| Vitor |
| Posted: Mon Mar 15, 2010 9:27 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
I'll also accept (if only for my own piece of mind) anecdotal evidence. War stories of the time someone accidently enabled an event and brought down production, or anyone who's been using 100s of events for years without problems.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Mar 15, 2010 11:41 am Post subject: Re: Overhead of performance events |
|
|
Poobah
Joined: 15 May 2001
Posts: 7717
|
| Vitor wrote: |
"any halfway decent WMQ admin knows better than to use events".
|
I guess the guys in Hursley that implemented it are only a quarter decent? 
Events kinda depend on the situation. Turn in Inhibit events, Inhibit every queue but one, have 1000 apps try to get and put as fast as possible to each inhibited queue, and see how the app using the one enabled queue behaves.
I haven't used Queue Service Interval events, but have used lots of others.
I guess it also depends on how tight the app's SLAs are. One app may die with a 10 ms delay per transaction. Another may not care at all.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Mar 15, 2010 12:27 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
Also keep in mind that the delay may have side effects on capacity.
Say your average response time is 20 ms and your SLA is 50 ms. You implement events and that moves your average response time to 40 ms... you'll say what still in the SLA... Sure but you may just have cut your throughput in 2.....
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Mon Mar 15, 2010 3:08 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9396
Location: US: west coast, almost. Otherwise, enroute.
|
There's not all that much computing to be done for service interval events, just clock time end - clock time start; then compare the result to your threshold specification. If your specification has been exceeded, then an event message gets created.
To begin gathering some perf data, perhaps the clients admins will tolerate enabling this event for 1 minute every hour.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| mvic |
| Posted: Mon Mar 15, 2010 5:58 pm Post subject: Re: Overhead of performance events |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| Vitor wrote: |
"any halfway decent WMQ admin knows better than to use events".
This position is a little light on evidence |
Ensure the app draining the event queue is reliable. If that queue does get deep (eg. because the app draining it is not reliable), it could become burdensome to load/unload. Ensure events are not being generated more quickly than can be consumed off the event queue.
Keep to all of the above, and IMHO you should get more benefit than cost out of event messages. |
|
| Back to top |
|
|
| mvic |
| Posted: Mon Mar 15, 2010 6:01 pm Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| fjb_saper wrote: |
| Say your average response time is 20 ms and your SLA is 50 ms. You implement events and that moves your average response time to 40 ms... |
Are those real numbers from a test? In that scenario are there several events per application message? |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Mar 15, 2010 6:14 pm Post subject: Re: Overhead of performance events |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| PeterPotkay wrote: |
| Vitor wrote: |
"any halfway decent WMQ admin knows better than to use events".
|
I guess the guys in Hursley that implemented it are only a quarter decent? |
We're all just a bunch of idiots it seems. I personally feel crushed... 
I thank everybody for the input. The intention here is to use QSI to ensure the app draining the queue isn't asleep at the switch. Or flat on it's back.
So I don't expect a large volume of event messages.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Mar 15, 2010 8:47 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
| mvic wrote: |
| fjb_saper wrote: |
| Say your average response time is 20 ms and your SLA is 50 ms. You implement events and that moves your average response time to 40 ms... |
Are those real numbers from a test? In that scenario are there several events per application message? |
No, not real numbers... this is just an example to illustrate the side effect.
We have had a few side effects like that on the broker, mainly created by a degradation of the response time from the service, due to some batch process running on the service....
On a normal day, the increase is no sweat. On peak time it can nearly kill you if you rely on your normal average... as it throttles throughput more effectively than MQ or broker capacity... especially in a request reply scenario. (from 200 to 400 ms but you still get the picture)...
As a real world experience, we have had events running on a qmgr in QA with nearly no perceptible performance impact but for the fact that we needed to run something like MQExplorer to get the messages to empty the event queue.... way more often than anticipated.... Writing an event message for every possible event that can be monitored by the qmgr will get old fast... You'd be better off with decent monitoring software.
The biggest danger here would be to try and centralize the event queue across the landscape and end up interfering with channel throughput if the app consuming the events does not keep up and the event queue is not deep enough...
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Fri Mar 19, 2010 11:16 am Post subject: Re: Overhead of performance events |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3253
Location: London, ON Canada
|
| Vitor wrote: |
| This position is a little light on evidence apart from anecdotal evidence that she was at a tech conference once and everyone was agreeing that Events Kill Queue Managers, but she has the advantage of position and credibility. |
I have been to a number of MQ Conferences and I have never heard that.
| Vitor wrote: |
| "won't support that additional load" |
The first thing I would say is prove that statement. Does your company save the server monitoring data for at least a month? If so, then pick a day and then turn on events and watch it for a month. After a month, look at the historical data and the new data from the past month. Better, worst or the same?
This is the only real way to quantify if event messages will add any load to your particular server(s).
As someone once said "show me the money!!".
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| zpat |
| Posted: Fri Oct 28, 2011 6:39 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5849
Location: UK
|
What's the impact (overhead) of turning on queue monitoring (MONQ)?
I know that the low, medium, high setting has the same effect - but what is the overall effect for an entire queue manager likely to be?
I know (before someone says) "it depends". I want to know what actually happens when this is done, and then I can consider the probable impact.
Anyone done any measurements on this? |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Oct 28, 2011 7:54 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9396
Location: US: west coast, almost. Otherwise, enroute.
|
(Obviously) more data sampling takes place; which means more cpu cycles. I've not seen stats comparing none, low, medium, high, and impact on throughput.
If stats are needed, it's a required overhead. I'm in favor of always capturing stats - test, qa, prod, to do capacity planning and p-d.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| zpat |
| Posted: Fri Oct 28, 2011 8:51 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5849
Location: UK
|
On MONQ - High, Medium or Low makes no difference.
It is either on or off in effect. But what actually is being done by MQ?
Extra I/O per get/put, or not? Much difference if using persistent messages? |
|
| Back to top |
|
|
| zpat |
| Posted: Mon Oct 31, 2011 12:02 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5849
Location: UK
|
I did some basic measurements, using the tools in IH03.
No effect on persistent messages (since they are already I/O bound).
17% reduction in non-persistent message throughput (presumably due to the extra i/o).
Of course this figure relates to my test setup, message sizes etc and will vary. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Mon Oct 31, 2011 3:28 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9396
Location: US: west coast, almost. Otherwise, enroute.
|
Monitoring is data gathering, and this is done in virtual storage, and takes cpu cycles... and not all that many of them Creating event messages takes i/o's.
It may take a bit of analysis to pick thresholds so as not to generate one i/o for every data gathering instance.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
|
|
