| Author |
Message
|
| ruimadaleno |
 Posted: Fri Dec 05, 2014 6:47 am Post subject: message flow statistics - best practice |
 |
|
Master
Joined: 08 May 2014
Posts: 274
|
Hi all,
we have enabled message flow statistics in our production environment (after enabling in QUA and DEV environnment). So far we don't hit performance issues due to enabling statistics.
I've read some documentation on the subject and asked google friend but i need a hand from who have walked this path before.
Now i'm looking for best way to check/visualize and/or monitor message flow statistics generated so i can get a picture on the message flows resource consumption on this environment.
i'd like to ear from you, experiencied and illuminated people  , what's the best way to gather/visualize/monitor message flow statistics
right now, for starting, i'm checking the statistics throughout message broker explorer ... what are, in your opinion, the cons and pros of this initial approach ?
My initial goals:
Understand which message flows are using more CPU
Understand which message flows are receiving more messages
Understand how muck time flows are using to execute their functions (i understand this can by hard to get from message flow statistics)
environnment:
WBM 8.0.0.4 running on windows 2008 server R2 enterprise
_________________
Best regards
Rui Madaleno |
|
| Back to top |
|
 |
| Vitor |
| Posted: Fri Dec 05, 2014 7:04 am Post subject: Re: message flow statistics - best practice |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| ruimadaleno wrote: |
| i'm checking the statistics throughout message broker explorer ... what are, in your opinion, the cons and pros of this initial approach ? |
Pros:
- easy to set up
- easy to understand
Cons:
- completely manual
- limited historical trending or perspective
- no alerting capability
| ruimadaleno wrote: |
i'd like to ear from you, experiencied and illuminated people , what's the best way to gather/visualize/monitor message flow statistics
|
We consume the XML using a flow and add it to a database. Interested parties use views or their own custom queries to load whatever they're interested in to Excel and draw graphs and charts to their heart's content. We were going to put a proper web based dashboard on the front of this, but ran out of time, money and interest.
The flow that does the load also has a series of criteria coded into it; if any of these are met it emails the support inbox, if the support inbox gets an email from the Prod flow it pages (it's our standard support mechanism - things email support and the inbox applies rules automagically somehow).
Future enhancements - when we get some free time on IIBv9 we're going to swap the criteria for a descision service, or we're going to plug these statistics into the shiny new CA monitoring tool the site has bought (which is awesome at monitoring Java in WAS but doesn't know WMB or IIB from a hole in the ground) and let that apply criteria.
My 2 cents, other solutions are equally valid and may be better for your individual requirements and no liability, express or implied, is accepted for any loss or damage howsoever caused as a result of actions taken as a consequence of these comments or otherwise.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Dec 06, 2014 3:32 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Don't reinvent the wheel. Make a list of the must have criteria, check out the commercially available monitoring software, and choose with a mind to ease of configuration and maintainability....
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| McueMart |
| Posted: Mon Dec 08, 2014 1:50 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
For prior to IIB v9 (when you get it built into the web interface), check out IS03 http://www-01.ibm.com/support/docview.wss?uid=swg24030716
From what you describe it'll do most of what you are after. Note it wasn't designed be to a long running monitoring tool - but rather a performance analysis tool. |
|
| Back to top |
|
|
| ruimadaleno |
| Posted: Tue Dec 09, 2014 4:48 am Post subject: |
|
|
Master
Joined: 08 May 2014
Posts: 274
|
Thank you all for you reply's, lots of info and new directions here ! 
fjb_saper: i'm going to look for comercial software, but from my knowledge, this kind of software is highly focused on queue monitor and less focused in message flow monitoring. Of course i need to monitor queue's behaviour, but right now the focus goes to message flow monitoring. Any tool you can point to me ?
Vitor: you implementation is a must , thanks for sharing. Can you point me some links on Knowledge Center on how to read message flow statistics , i understand mf stats are available in a queue, and i'd like to explore this further. We are using Check_mk
[url] http://mathias-kettner.com/check_mk.html [/url] to monitor the environments, maybe we can use ChecK_mk to consume this stats and monitor it. Are you aware of some implementation with check_mk reading message flow stats ?
MCueMart: i've already tried IS03 in Development and Staging environment. Right now we are configuring channels in production environment so we can use IS03 to check message flows performance.
_________________
Best regards
Rui Madaleno
Last edited by ruimadaleno on Wed Dec 10, 2014 2:12 am; edited 1 time in total |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Dec 09, 2014 5:09 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| ruimadaleno wrote: |
fjb_saper: i'm going to look for comercial software, but from my knowledge, this kind of software is highly focused on queue monitor and less focused in message flow monitoring. Of course i need to monitor queue's behaviour, but right now the focus goes to message flow monitoring. Any tool you can point to me ? |
Check with your known MQ monitoring suspects. Most have a broker monitoring module. Be sure to specifically ask for it to be demoed.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Tue Dec 09, 2014 1:15 pm Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| ruimadaleno wrote: |
| .... We are using Check_mk (http://mathias-kettner.com/check_mk.html) to monitor the environments, maybe we can use ChecK_mk to consume this stats and monitor it. ... |
I am not sure if that link is valid to explorer. (Links should be used via URL tag while posting.) 
| ruimadaleno wrote: |
| ... i've already tried IS03 in Development and Staging environment. Right now we are configuring channels in production environment so we can use IS03 to check message flows performance. |
It is a known Support Pack for Performance/Stats analysis at Message Flow level. Also, helps with historic view of the stats data. 
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Tue Dec 09, 2014 2:42 pm Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| Quote: |
Pros:
- easy to set up
- easy to understand
Cons:
- completely manual
- limited historical trending or perspective
- no alerting capability |
Just being curious...
In contrast, How do we setup alerting using IS03 ?
Can we increase the Statistics Interval to 1 or 5 minutes interval instead of default of every 20 seconds for Snapshot statistics, at IS03 ?
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Dec 09, 2014 2:46 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| IS03 is not a monitoring tool that can generate alerts or store statistics over long periods of time. |
|
| Back to top |
|
|
| ruimadaleno |
| Posted: Wed Dec 10, 2014 2:18 am Post subject: |
|
|
Master
Joined: 08 May 2014
Posts: 274
|
| mqjeff wrote: |
| IS03 is not a monitoring tool that can generate alerts or store statistics over long periods of time. |
i agree, IS03 is a good tool to check performance issues in realtime, in a reactive fashion, ie, check performance issues when you get an high cpu usage by dataflowengine.exe process and need to understand what message flow and what piece of that message flow is consuming resources. Is has no capibility to send any kind of alerts , it doesn't store statistics , so it doesn't allow you to analyze it later.
_________________
Best regards
Rui Madaleno |
|
| Back to top |
|
|
| McueMart |
| Posted: Wed Dec 10, 2014 2:50 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
IS03 does store the stats to allow you to export them later (in a csv format). But it's not designed to run for days at a time collecting them. If you leave it running for more than a day or 2, your JVM will probably run out of memory (all the stats data is held in memory).
| Quote: |
| Can we increase the Statistics Interval to 1 or 5 minutes interval instead of default of every 20 seconds for Snapshot statistics, at IS03 ? |
Take a look at Archive statistics and remember to have IS03 listen on the archive statistics publication.
http://www-01.ibm.com/support/knowledgecenter/SSMKHH_9.0.0/com.ibm.etools.mft.doc/ac19102_.htm |
|
| Back to top |
|
|
|
|
