| Author |
Message
|
| AkankshA |
 Posted: Tue May 19, 2009 8:44 pm Post subject: Monitor Messages Flowing Through SVRCONN |
 |
|
Grand Master
Joined: 12 Jan 2006
Posts: 1494
Location: Singapore
|
Hi,
I have my application deployed on W application server and connecting to MQ MQ through client connection.
Both MQ and application reside on same machine....
I intend to capture each and every message flow flowing from application to MQ.. along with time stamp (which ll eventually be available with MQMD) and payload....
Possible way to get that is using MQ channel exit.... i intend to use receive exit for the same...
I though about using every trigger to start a process which ll capture the timestamps only, but i cant since the messages come to an ALias queue which in turn gets load balanced in cluster... hence triggering is ruled out
gimme ideas wrt same...
is there any other way out....
also any sample code available for the exit ??
_________________
Cheers |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue May 19, 2009 11:33 pm Post subject: Re: Monitor Messages Flowing Through SVRCONN |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| AkankshA wrote: |
| I intend to capture each and every message flow flowing from application to MQ.. along with time stamp (which ll eventually be available with MQMD) and payload.... |
Why? There's been enough debate in here about auditing messages in WMQ rather than in the end points. What's the requirement? Especially with client monitoring, and when the client is in such proximity.
The problem of exits (which I still hold to be tricky despite one noteable voice to the contrary) is compounded by use in a cluster.
| AkankshA wrote: |
| is there any other way out.... |
Capture the information in the end point systems
| AkankshA wrote: |
| also any sample code available for the exit ?? |
I'm sure a volunteer will be delighted to provide some momentarially.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| AkankshA |
| Posted: Wed May 20, 2009 12:39 am Post subject: |
|
|
Grand Master
Joined: 12 Jan 2006
Posts: 1494
Location: Singapore
|
well,
the strange requirement arose owing to the delay experienced in processing.
People are doubting that MQ routing in a cluster is taking time...  I need to give them facts to convince that its application's problem and not MQ.... that its application which is not committing or rather putting messages at designated time with LOAD
so i have no way but to show them, at what time a particular message arrived in the queue....
_________________
Cheers |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed May 20, 2009 1:04 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| AkankshA wrote: |
| People are doubting that MQ routing in a cluster is taking time... |
I see your problem.
| AkankshA wrote: |
| so i have no way but to show them, at what time a particular message arrived in the queue.... |
An alternative (of debatable value I agree but an alternative) is to monitor the depth of the SCTQ. If the cluster is taking time to move messages round, you should see messages piling up there.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| AkankshA |
| Posted: Wed May 20, 2009 1:23 am Post subject: |
|
|
Grand Master
Joined: 12 Jan 2006
Posts: 1494
Location: Singapore
|
| Vitor wrote: |
An alternative (of debatable value I agree but an alternative) is to monitor the depth of the SCTQ. If the cluster is taking time to move messages round, you should see messages piling up there. |
I told/shown them that... but non MQ aware people find it difficult to believe that MQ can be so fast....
so i hv two options
1) MQ Receie exit for SVRCONN channel (nightmare to write and set up)
2) Trigger SCTQ for every type and qrite a shell script which will log the timestamp in a file (alas, I cant get the message here)
Which one do you suggest would be better for performance... obviouly they wont be kept on for forever...
its difficult to choose a viable option amongst the two wrong/difficult ones...
_________________
Cheers |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed May 20, 2009 1:47 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| AkankshA wrote: |
| I told/shown them that... but non MQ aware people find it difficult to believe that MQ can be so fast.... |
There's never a trout to hand when you need one is there? 
| AkankshA wrote: |
| its difficult to choose a viable option amongst the two wrong/difficult ones... |
I think you've put your finger on it when you said "viable". IMHO an exit is never the "right" answer, and taking my prejudice to one side, I think it will be more work to set up and test than is warrented by your requirement.
Have you considered using queue service interval events to prove things are moving? I accept that in this situation non-MQ people may not believe evidence produced by WMQ, but then will they believe a triggered process or an exit? At some point they must accept.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed May 20, 2009 1:57 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Taking a step back from the problem here, another possible tack is to use a lack of evidence. This "delay in processing" - how did the finger get pointed at WMQ? If it's just "well our application is perfect so it must be WMQ because it's always WMQ just like when our messages go missing" then push back and ask how they came to this conclusion.
If they have proof, e.g. the time stamp they committed the message v the time it was read off the destination queue, then possibly you could use trace messages or similar. At least you'll have something to work with.
If they just have a profound belief that it can't be their application code because it just can't be, ask for proof. Or for trout.
My 2 cents.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| KeeferG |
| Posted: Wed May 20, 2009 2:53 am Post subject: |
|
|
Master
Joined: 15 Oct 2004
Posts: 215
Location: Basingstoke, UK
|
I have to agree with Vitor. You need to push this back to the development team and make them provide the evidence.
Too many times I have been involved with development teams that assume their code is right and it is the product at fault. Just recently I have had to bring IBM in for performance discussions over a message latency issue that we had been having and that development had been pointing the finger at MQ.
Fortunately, a recent update to the development code removed 90% of the applicaiton latency which cleared MQ in the eyes of development.
I would advise running end to tests with traceroute to see what results that consistently gives. If that is greatly quicker then the speed they are experiencing then they need to revisit their code.
_________________
Keith Guttridge
-----------------
Using MQ since 1995 |
|
| Back to top |
|
|
| AkankshA |
| Posted: Wed May 20, 2009 2:58 am Post subject: |
|
|
Grand Master
Joined: 12 Jan 2006
Posts: 1494
Location: Singapore
|
well, its a heterogeneous env.... and MQ is connecting them all...
the time gap between sending application and the receiving one is crossing the SLA.....
There's no exact way to know when the sending application put/committed the message to MQ... the problem increases manyfold, when we hv load...
may be, the application is not doing its job well, and hence message is not getting put... so we gotta capture the time stamp of each and every message coming to the designated queue and compare it with the receiving application timestamp...
as of now, i am thinking of putting every trigger on SCTQ, and if problem persists for longer then will implement channel exits..
_________________
Cheers |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed May 20, 2009 3:12 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| AkankshA wrote: |
| may be, the application is not doing its job well, and hence message is not getting put... so we gotta capture the time stamp of each and every message coming to the designated queue and compare it with the receiving application timestamp... |
Erm....if the message isn't being put, how will you capture the timestamp? 
You'll also need to capture enough message data so that you can tie up business process within the applications (unless the 2 ends can be persuaded to report the MsgId of the messages involved), i.e. Order 1234 is put at Time X, is captured on the queue at Time X+1, is delivered to the target queue at Time X+5, and processed by the receiving app at Time X + 14 minutes....
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed May 20, 2009 4:45 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Really, I think trace messages are the answer here.
You can show them how long it takes MQ to move messages over their exact route, and then say "if your applicaiton takes 10% longer, it's your application's fault". |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed May 20, 2009 8:51 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| AkankshA wrote: |
... the problem increases manyfold, when we hv load...
|
Is this a JMS app doing selective gets against queues with a depth > 0?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed May 20, 2009 1:05 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| PeterPotkay wrote: |
| AkankshA wrote: |
... the problem increases manyfold, when we hv load...
|
Is this a JMS app doing selective gets against queues with a depth > 0? |
Do you get backlog on the xmitq during high load?
The problem might be an application not processing fast enough...
This could potentially affect the whole cluster but will certainly affect all messages going to and possibly coming from that node.
The problem alluded to by Peter would then just compound...
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| AkankshA |
| Posted: Wed May 20, 2009 8:35 pm Post subject: |
|
|
Grand Master
Joined: 12 Jan 2006
Posts: 1494
Location: Singapore
|
| mqjeff wrote: |
Really, I think trace messages are the answer here.
You can show them how long it takes MQ to move messages over their exact route, and then say "if your applicaiton takes 10% longer, it's your application's fault". |
I did thought about Tracing, but then the amount of logs which gets generated is huge and fetching required information again wont be easy.... considering the heavy load situation...
_________________
Cheers |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu May 21, 2009 12:04 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| AkankshA wrote: |
| I did thought about Tracing, but then the amount of logs which gets generated is huge and fetching required information again wont be easy.... considering the heavy load situation... |
Not trace, but trace messages.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
