Author |
Message
|
hughson |
Posted: Tue May 30, 2017 2:14 am Post subject: What can you check with Activity Trace |
|
|
 Padawan
Joined: 09 May 2013 Posts: 1959 Location: Bay of Plenty, New Zealand
|
Has anyone thought of things you can check about applications with MQ's Activity Trace. You may have noticed that MQGem's MO71 is now analyzing Activity Trace. There are a few things we've thought of to highlight if we see it in the trace, but was wondering if anyone had other ideas, or things that they've used Activity Trace for that would fall into this category that they'd like to see a trace analyzer point out.
Here's the current list:-
- Not using 'Fail if Quiescing'
- MQGET not using MQGMO_CONVERT
- Message sizes too big or too small
- Not using any Syncpoint option
- MQGET using small wait interval
- MQPUT using default persistence
- MQPUT using no context
Cheers,
Morag _________________ Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
Back to top |
|
 |
PeterPotkay |
Posted: Tue May 30, 2017 12:10 pm Post subject: |
|
|
 Poobah
Joined: 15 May 2001 Posts: 7722
|
Using Unlimited Expiry and Not Persistent at the same time...."Do you want the message to live f-o-r-e-v-e-r, or don't you?"
Tiny Expiry values... "I told the messages to live 10 seconds, see, look, I coded '10' for Expiry."
Truncated message warnings..."My MQ call didn't fail, see, I check for MQCC != 2. Why didn't you send me the whole message?!"
Not specifying the MQMD Format
Not specifying the MQMD Type
Not specifying the MQMD Priority _________________ Peter Potkay
Keep Calm and MQ On |
|
Back to top |
|
 |
Vitor |
Posted: Tue May 30, 2017 12:15 pm Post subject: |
|
|
 Grand High Poobah
Joined: 11 Nov 2005 Posts: 26093 Location: Texas, USA
|
PeterPotkay wrote: |
Using Unlimited Expiry and Not Persistent at the same time...."Do you want the message to live f-o-r-e-v-e-r, or don't you?"
|
The "hang on as long as you can" option....... _________________ Honesty is the best policy.
Insanity is the best defence. |
|
Back to top |
|
 |
PaulClarke |
Posted: Tue May 30, 2017 2:07 pm Post subject: |
|
|
 Grand Master
Joined: 17 Nov 2005 Posts: 1002 Location: New Zealand
|
The problem with a small expiry is that as a message travels through the system it's expiry gets smaller and smaller. So, it is entirely possible that it started off a decent size
and yet we do end up putting a small expiry quite legitimately. At least that is the reason we didn't check it. However, thinking again I suppose the likelihood is still pretty small.
Especially in a controlled testcase.
Thanks for the suggestion, I may well add Expiry. _________________ Paul Clarke
MQGem Software
www.mqgem.com |
|
Back to top |
|
 |
mqjeff |
Posted: Wed May 31, 2017 3:37 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008 Posts: 17447
|
PaulClarke wrote: |
The problem with a small expiry is that as a message travels through the system it's expiry gets smaller and smaller. |
I was holding onto that cliff edge with two hands, why am I using only one finger now? _________________ chmod -R ugo-wx / |
|
Back to top |
|
 |
PeterPotkay |
Posted: Wed May 31, 2017 6:05 am Post subject: |
|
|
 Poobah
Joined: 15 May 2001 Posts: 7722
|
PaulClarke wrote: |
The problem with a small expiry is that as a message travels through the system it's expiry gets smaller and smaller. So, it is entirely possible that it started off a decent size
and yet we do end up putting a small expiry quite legitimately. At least that is the reason we didn't check it. However, thinking again I suppose the likelihood is still pretty small.
Especially in a controlled testcase.
Thanks for the suggestion, I may well add Expiry. |
Please do, I think it will be very useful.
It will help identify originating apps that give birth to a message with a small expiry.
And even if the message started with a reasonable expiry, as it traverses the MQ toplogy and that expiry gets smaller and smaller and smaller. would be nice to have MO71 flag the point where it gets critically small.
"Small" Expiry is extremely relative, so hopefully if you allow for this you will also allow the MO71 user to determine the expiry value to flag. _________________ Peter Potkay
Keep Calm and MQ On |
|
Back to top |
|
 |
PaulClarke |
Posted: Tue Jun 06, 2017 4:10 pm Post subject: |
|
|
 Grand Master
Joined: 17 Nov 2005 Posts: 1002 Location: New Zealand
|
Thank-you all for your suggestions. I have just put up the latest Beta driver of MO71 9.0.4 which contains a number of them. This driver also contains a much requested feature which is the ability to import an MQ Explorer location export file.
You can download the Beta from http://www.mqgem.com/beta_downloads/mo71_9.0.4_Beta.zip.
As always I am interested in any comments you may have,
Cheers,
Paul. _________________ Paul Clarke
MQGem Software
www.mqgem.com |
|
Back to top |
|
 |
tczielke |
Posted: Fri Jun 30, 2017 6:58 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010 Posts: 941 Location: Illinois, USA
|
Personally, I have had to move away from using the Activity Trace. At least in a Production or Test environment. I have been bit one too many times now on both Unix and Windows environments where turning the Activity Trace on causes the queue manager agent (amqzlaa0) to receive an internal error (e.g. SIGSEGV) and cause the queue manager to become unstable. The queue manager then has to be recycled to restore service. It is a rare occurrence for this to happen, but I can't be turning this tool on with the rare but very possible reality that it might break the queue manager. Maybe others have had different experiences, but personally I have been snake bitten one too many times.  _________________ Working with MQ since 2010. |
|
Back to top |
|
 |
PaulClarke |
Posted: Fri Jun 30, 2017 7:31 am Post subject: |
|
|
 Grand Master
Joined: 17 Nov 2005 Posts: 1002 Location: New Zealand
|
I hope you raised a PMR with IBM Service....and of course that they fixed it
I have not heard of such issues, nor did we experience any such problems during the Development of this feature, but in the software game there is always a 'risk' to use any feature, especially when it is new. Assuming that service is pretty good, and I believe MQ's is, the problems tend to get ironed out pretty quickly though.
If you don't want to use Activity Trace that is, of course, entirely your prerogative. However, it seems preferable to me that running a home-grown solution using API exits or whatever. That seems to attach a much greater risk to me. It would seem a shame for people to miss out on a useful feature of the IBM product, whether they choose to exploit it using MO71 or not.
Cheers,
Paul. _________________ Paul Clarke
MQGem Software
www.mqgem.com |
|
Back to top |
|
 |
tczielke |
Posted: Fri Jun 30, 2017 7:53 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010 Posts: 941 Location: Illinois, USA
|
I have raised PMRs in the past, and sometimes IBM has been able to provide PTFs for them, and sometimes not. I do suspect there are still lingering bugs in the Activity Trace code.
The Activity Trace does work without these issues over 99% of the time, but I have had 3 occurrences over the past few years (and one this week on Linux) where I caused a Production incident by just turning the tool on.
I hate to be at this point, but I can't in good faith turn on a diagnostic tool that has the potential (albeit rare) to cause the queue manager to become unstable.
Personally, I have moved back to just using strmqtrc tracing. With the MH06 supportpac, I can basically get what I was getting with the Activity Trace.
I invested a lot of time into enhancing amqsact, so I don't make this deviation lightly. But again, I just can't risk causing Production issues by turning on a diagnostic tool. _________________ Working with MQ since 2010. |
|
Back to top |
|
 |
PaulClarke |
Posted: Fri Jun 30, 2017 7:59 am Post subject: |
|
|
 Grand Master
Joined: 17 Nov 2005 Posts: 1002 Location: New Zealand
|
Well, as I said before, it is impossible to use anything without an element of risk. I seem to remember a number of issues with strmqtrc where switching it on in certain circumstances caused the process to abend.
As we all know, writing good bug-free software it almost impossible. Some would argue that it is impossible and that there are always bugs lurking. But without doubt the way to improve the quality of the code is to run it and fix it as you go. As the saying goes, hardware eventually fails, software eventually works!
Cheers,
Paul. _________________ Paul Clarke
MQGem Software
www.mqgem.com |
|
Back to top |
|
 |
PeterPotkay |
Posted: Fri Jun 30, 2017 4:26 pm Post subject: |
|
|
 Poobah
Joined: 15 May 2001 Posts: 7722
|
The more people that use something, the more likely bugs are found, reported and fixed.
Its likely Paul's enhancement to MO71 as it relates to App Activity trace will accomplish that for App Activity trace.
Tim, the issues you had, were they restricted to the moment you flipped the bit to tell the QM to start App Activity tracing? Or did you see things seemingly stable for minutes / hours / days and only then it started acting up? _________________ Peter Potkay
Keep Calm and MQ On |
|
Back to top |
|
 |
tczielke |
Posted: Fri Jun 30, 2017 4:38 pm Post subject: |
|
|
Guardian
Joined: 08 Jul 2010 Posts: 941 Location: Illinois, USA
|
When I have experienced the issue, it usually happens fairly quickly to when the application activity trace was turned on. I turn it on with an "ALTER QMGR ACTVTRC(ON)".
The first time I had this issue it was on Windows and had to do with an mqat.ini that I had copied over from Linux and did not have the format of how Windows like text files line to end with "\n\r". IBM was able to provide a fix for that, and I also had a workaround with correcting the mqat.ini file format.
The second time I had this issue was with Windows again, and IBM was not able to diagnose the issue. I just had the FDC and that was not enough for them to diagnose it.
The third time I had the issue (this week) was on Linux. I have an open PMR for this last occurrence.
I know software can come with bugs, but this is an area where IBM really needs to be better. You just can't have the queue manager falling apart like this! _________________ Working with MQ since 2010. |
|
Back to top |
|
 |
tczielke |
Posted: Thu Jul 06, 2017 4:41 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010 Posts: 941 Location: Illinois, USA
|
FYI - IBM was able to use the FDC to diagnose the latest issue I encountered with the Activity Trace on Linux. The future APAR that will address it is IT21300. _________________ Working with MQ since 2010. |
|
Back to top |
|
 |
tczielke |
Posted: Tue Sep 05, 2017 10:09 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010 Posts: 941 Location: Illinois, USA
|
For those interested in APAR IT09496, it has closed and here are the tentative fixpacks it will be released in.
Code: |
Version Maintenance Level
v7.5 7.5.0.9
v8.0 8.0.0.8
v9.0 CD 9.0.4
v9.0 LTS 9.0.0.3
|
_________________ Working with MQ since 2010. |
|
Back to top |
|
 |
|