| Author |
Message
|
| smdavies99 |
 Posted: Sat Jun 12, 2010 10:22 pm Post subject: Monitoring Messages |
 |
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
I'm having a go at understanding the data in both FlowStats & Monitoring messages.
I've written a flow to process the start/end messages and calc the elapsed times.(runs in a different EG to the flows being monitored)
One thing has puzzled me is that in about 10% of the cases and when the elapsed time of the flow being monitored in very short (1-2mSecs) then I'm seeing the 'transactionEnd' message on the queue before the 'transactionStart' message
Environment
Windows Server 2003 SP2.
Broker : 6.1.0.5
Has anyone got an explanation for this? (before I raise a PMR...  )
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Sun Jun 13, 2010 1:36 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
I am a tad puzzled... What is your statistics publication/collection interval? Is your broker flow also raising events? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Sun Jun 13, 2010 4:37 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Sorry FJB, I might have confused you.
These are not the FlowStat but the Monitoring events.
mqsireportflowmonitoring etc
There are TWO EG's on my test broker.
The first EG has the flow I'm using to generate the events.
The second EG has the flow that consumes the events.
Flow Monitoring is only enabled for the first EG.
Monitoring event are produced immediately the event occurs.
Then when the MQInput Node picks up the message, the 'transactionStart' event is generated.
When the thread dies either a 'transactionEnd' or 'transactionRollback' event is generated.
If I coulds guarantee that the 'Start' was always put onto the subscription Queue before the 'end', the logic in the consumer flow would be a lot simpler.
As they do appear 'out of order' when the elapsed time is very small, I have to do all sorts of 'stuff' in the flow to match the event_id's up properly and calculare the elapsed time.
Does this further explanation help?
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Jun 13, 2010 7:06 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
If my memory serves, events are being published.
It seems that somewhere in the pub/sub manual there was something about being able to sequence publishes and how to do it (can't quite remember where I saw it, just that I saw it...). I fear you might have to look into this. Don't know however if this is possible using the event generation interface on the flow...
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqjeff |
| Posted: Sun Jun 13, 2010 9:09 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| If you know that you will receive six discrete events that you need to match up based on a unique id and you need to process them in order, you might look at Collector. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Sun Jun 13, 2010 10:20 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| mqjeff wrote: |
| If you know that you will receive six discrete events that you need to match up based on a unique id and you need to process them in order, you might look at Collector. |
Six events eh?
I've only ever seen Three with the FlowMonitoring events. (start, end, rollback).
That said, the collector node should help with what I'm trying to acheive in the flow. I'd forgotten that one.(Sigh)
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Sun Jun 13, 2010 3:28 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| smdavies99 wrote: |
| mqjeff wrote: |
| If you know that you will receive six discrete events that you need to match up based on a unique id and you need to process them in order, you might look at Collector. |
Six events eh? |
That was an entirely random number based on absolutely nothing. Do not take it as anything other than that. |
|
| Back to top |
|
|
| Mandeep |
| Posted: Mon Jun 14, 2010 3:00 am Post subject: |
|
|
Apprentice
Joined: 03 May 2004
Posts: 33
|
There are in and out terminal level events apart from the Start, End and Rollback ones.
I tried to use events as an alternative for logging (log4j), but had to give up coz of order of events published could not be gauranteed.
Though there is a sequence assigned to them, but it became more complex as I had more than 1 number of threads deployed for the same message flow and then the events published to a Queue had to be identified based on thread's unique id as well. |
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Jun 14, 2010 4:13 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| I tried to use events as an alternative for logging |
That's not really what they are meant for. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Mon Jun 14, 2010 4:28 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
The Terminal Events are optional.
I'm looking at using the Flow Start/Stops times to track overall elapsed times in a multi-tiered SOA.
More detailed analysis can be made using FlowStats on individual services (aka Flows) when required.
We want to rip out all the legacy stuff (from 2.1/5.0 days) that relied on Developers following the guidelines and adding code to their flows/subflows to record the times. This obviously slows down the execution of the services by at least 10% due mainly to the start/end flow overheads.
I've used the collector node and so far it is working fine. I'll be giving it a really hammering later today with a set of automated tests (some 100K) messages.
My only criticism of the FlowMonitoring is that you have to remember to enable it after the deployment of a new flow. So our deployment scripts will need modifying to do this as a matter of course.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| ghoshly |
| Posted: Tue Jun 15, 2010 6:54 am Post subject: I also wanted it to be the other way around |
|
|
Partisan
Joined: 10 Jan 2008
Posts: 333
|
Thats right. It could have been the other way round, i.e we are adding events only because we need them (Terminal levels), so by default the monitoring should have been active.
If anyone does not wants for better performance or so can disable them, as a part of tuning. |
|
| Back to top |
|
|
| prasad.edlabadkar |
| Posted: Wed Jun 16, 2010 5:17 am Post subject: Re: Monitoring Messages |
|
|
Novice
Joined: 10 Feb 2006
Posts: 21
Location: Pune, India
|
| smdavies99 wrote: |
I'm having a go at understanding the data in both FlowStats & Monitoring messages.
I've written a flow to process the start/end messages and calc the elapsed times.(runs in a different EG to the flows being monitored)
One thing has puzzled me is that in about 10% of the cases and when the elapsed time of the flow being monitored in very short (1-2mSecs) then I'm seeing the 'transactionEnd' message on the queue before the 'transactionStart' message
Environment
Windows Server 2003 SP2.
Broker : 6.1.0.5
Has anyone got an explanation for this? (before I raise a PMR... ) |
Hi,
If you want to measure the performance of a message flow, try using Support pack IS02 (http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg24012457). It uses accounting and statistics to measure the performance and presents them in a nice graphical format.
If you want to achieve the same thing using monitoring events, I suggest you simply write the events to DB as they arrive and use the timestamp from the event message to sort them.
_________________
Thanks and Regards
Prasad Edlabadkar
Tech Lead - Websphere Business Integration |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Wed Jun 16, 2010 6:42 am Post subject: Re: Monitoring Messages |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| prasad.edlabadkar wrote: |
If you want to achieve the same thing using monitoring events, I suggest you simply write the events to DB as they arrive and use the timestamp from the event message to sort them. |
This is exactly what I have done. I use a Collector Node to separate the start/end messages and then use the COLLECTION... output to populate a DB table with the start & stop time plus the elapsed time as a FLOAT.
It has processed over 1M messages so far without issue.
I've also built in a Timer driven bit that will delete Old events from the file.
This will stop the table from becoming humungous.
I'm doing the same sort of thing with Flow Stats.
Yesm IS02 could be the answer but mere mortals like me don't have WMQExplorer access to our Production Systems.
The DB tables are not on the PRoduction System so acceess is not an issue.
The processing of the data will only happen during the night so the only overhead during the working day is the writing of the data to the Publication Queue.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
|
|
