| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Message Tracking |
« View previous topic :: View next topic » |
| Author |
Message
|
| RogerLacroix |
 Posted: Wed Apr 28, 2004 7:37 pm Post subject: Message Tracking |
 |
|
Jedi Knight
Joined: 15 May 2001
Posts: 3264
Location: London, ON Canada
|
All,
I have tried to talk my current client into buying a message tracking product but of course they say they don't have the money!?!?! 
The problem is that the client has a lot of MQ development going on with a lot of newbie MQ/Java developers. And of course the newbie developers keep telling me that MQ lost their messages. This of course is driving me nuts!!! 
So I figured that I would create an API Exit to log the following:
- Queue Name
- Date / Time
- MsgID
- CorrelID
- GroupID
From a tracking point of view, I don't think logging the message data is important but what other fields of the MQMD should be logged??
I figure I would use Perl or Java to summarize or correlate the information in the log file. Of course, the script would cross search between MsgID, CorrelID & GroupID for matches.
Any comments - thoughts about this.
Regards,
Roger Lacroix |
|
| Back to top |
|
 |
| vijiraghav |
| Posted: Wed Apr 28, 2004 11:26 pm Post subject: Message Tracking |
|
|
Novice
Joined: 11 Nov 2003
Posts: 18
|
Dear Roger,
Excellent idea.
MQ will even be blamed if there is any problem/error/missing information in the message data. Hence it is better to log even the message also. I faced this problem, when the requesting application received response message in which some info./portion of data is missing (for which MQ is definitely not responsible but none would agree) and MQ was there naturally to take the blame.
In addition, you may also add the PutApplName and UserID fields to be logged.
Thanks and all the best.
vijiraghav |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Apr 29, 2004 3:53 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
If the message has an MQRFH2, it would be good to log that as well.
Or at least make an option to do so.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| mqonnet |
| Posted: Thu Apr 29, 2004 5:09 am Post subject: |
|
|
Grand Master
Joined: 18 Feb 2002
Posts: 1114
Location: Boston, Ma, Usa.
|
Roger, i was also following this topic thread on the listserv. One immediate answer that you could give to your client/developers is that, they need to *prove* to you that MQ is infact loosing messages. I know this might sound a bit harsh, but if i have a queue manager and a client connects to it and complains that they are loosing messages, and i cant find any relevant info on my side(logs, ffsts, events). Then the question that would follow is, how do they concur that the messages are infact being lost.
As you know MQ wouldnt ever loose a message unless there is either an app problem or MQ problem. In the latter, there would *surely* be some sort of logging some place. In the former... Hmm... Thats where i guess we need to concentrate. IMHO.
And i know i dont need to mentione all this, but just adding. If the messages are persistent and the put completes and commit occurs. There is *no* way MQ would loose the message without logging anything anywhere.
So, i would go back to the dev's and probably ask them to log their api calls outputs. Say after each put/get/commit/back log the reason code and correlate the same.
IMO its *not* the responsibility of the QM or MQ or the one managing the qm to wonder what happened to messages put by clients. Its the clients responsibility to log everything they do.
Again all this, IMO.
Cheers
Kumar |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Thu Apr 29, 2004 7:47 am Post subject: |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3264
Location: London, ON Canada
|
Hi,
I fully agree with you but sadly as the 'Messaging Architect' for one division I do not have the authority to request or mandate anything. I can only recommend things.
I have written WMQ Naming Standards documents and WMQ Programming Standards documents for the client but it just goes over the newbie's head.
When the newbies complain to their VP, who complains to my VP, who then complains to my manager and me being a consultant, you know what happens next!!!
So, this may be a difficult solution but it will give me fewer headaches. 
After various comments from the MQ ListServer, I think I should include all MQMD fields plus part of the message data (i.e. 100 bytes but make it configurable).
Regards,
Roger Lacroix |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Tue May 04, 2004 9:54 am Post subject: |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3264
Location: London, ON Canada
|
Oops, I forgot to update this posting. There was a large flurry of emails on the MQ ListServer with people expressing opinions from all sides.
So here is the current game plan: Since there appears to be interest in a product (one way or other), I have decided to create and market a new product called MQ Visual Tracker (or maybe MQ Visual Message Tracker). It will contain 2 separate but connected components: API Exit and a client Java Swing program
Api Exit:
(1) Installed on the queue manager
(2) Configuration information will be stored as a single message in a queue (i.e. API_EXIT_CTRL)
(3) Default logging would be all MQMD fields plus 100 bytes of data.
(4) Selectively enable / disable queues to be logged - stored in configuration message
(5) Real-time starting or stopping of the logging.
(6) Logging information would stored in a QUEUE (not a file) i.e. API_EXIT_LOG
Java / Swing program:
(1) Simple interface - point to a queue manager and show the API_EXIT_LOG queue
(2) Search function would allow the user to input a string or hex digits, then select which queue managers or all to do the search against.
(3) The search function would return a list of matches against MsgID, CorrelID and GroupID for all selected queue managers sorted by date.
(4) Therefore, you could see string 'XXXXX' start as a MsgID then transition to a CorrelID as the message flow from application to application via the various queue managers.
(5) Ability to edit / update the configuration message for each individual queue manager.
The one piece that is missing is a simple task to be run at midnight to do cleanup or archiving.
So feedback time. Thinking in terms of a light-weight solution, what other features are required?
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| Antony |
| Posted: Tue May 04, 2004 3:52 pm Post subject: |
|
|
Newbie
Joined: 22 Apr 2004
Posts: 5
Location: Melbourne, Australia
|
Roger,
Lets begin with the requirements;
From my experience I have found there are three main places logging is useful;
-in intergration testing (which app dropped the message?)
-when you have an immediate problem.
-when some problem is identified and filters back to you three months later.
I have found a database to be the most useful way to store logging data in meeting these requirements;
- It's something IT managers understand and are comfortable with.
- DBAs have already established processes for archiving and restoring data.
- Online data can be queried via a web interface (this is very useful for testers who's technical ability is not that advanced).
- SQL is mature search tool, which nothing availble for MQ really compares to.
Giving testers/application support the ability to query the log themselves via web tool or SQL, will relieve MQ support of a burden, by removing the spurious "MQ lost the message" reports we all receive. It is hard for me to see how this can be done well, without using a database as storage.
Antony |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue May 04, 2004 5:24 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Anthony -
What you're talking about is a little heavier weight than I think Roger wants to build.
But it's also very easy to add on top of his solution.
First off, one could simply not use his Java/Swing program at all, and put in your own program that would pull messages off the queue and put them in a database.
Additionally, one could write one's own program to BROWSE the queue, and put the messages into a database.
Or one could even install the mirrorq exit, and copy the messages from Roger's new queue to a different queue that's being consumed by your own program (or WMQI!  )
But I absolutely agree with everything you've said.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
