ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » IBM MQ Performance Monitoring » Application Activity Trace - Keeps going and going....

Post new topic  Reply to topic Goto page 1, 2  Next
 Application Activity Trace - Keeps going and going.... « View previous topic :: View next topic » 
Author Message
PeterPotkay
PostPosted: Tue Jul 22, 2014 5:49 am    Post subject: Application Activity Trace - Keeps going and going.... Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

MQ 7.5.0.2
I make a stanza in mqat.ini to watch ‘Websphere Datapower*’, turn the queue manager’s ACTVTRC property from OFF to ON and it captures what I want. Yay.

I then turn the queue manager’s ACTVTRC property from ON to OFF.

But the tracing keeps going and going.

Thinking it may be ‘stuck’ on tracing an active app with a still valid hConn, I stop the DataPower application (an MQ Client) and restart it, but the tracing continues.

Thinking it might be related to the amqrmppa process that the MQ Clients use, I stop the DataPower application, and other MQ Clients until I have no running channels. The amqrmppa process persists so I kill it (this is a lab machine ). I then restart the DataPower app, but App Activity Tracing keeps going and going, like an unstoppable zombie.

I turn the queue manager’s ACTVTRC property from OFF to ON to OFF to ON lobbing a few curse words every so often, but the tracing keeps going and going.

ACTVTRC property is OFF. I alter the stanza in mqat.ini to turn tracing off for ‘Websphere Datapower*’, but the tracing keeps going and going. Realizing that perhaps the QM only reads mqat.ini at start up or when turn the queue manager’s ACTVTRC property is toggled, I turn the queue manager’s ACTVTRC property from OFF to ON and finally the tracing stops. Phew!

Anyone else seen this or any documentation that talks about it? It doesn’t seem ACTVTRC set to OFF means off!
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Fri Aug 15, 2014 6:44 am    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

My PMR was closed today. APAR IT03760 was opened to document the nuances detailed below, because this info is currently not in the Knowledge Center nor in a Tech Note.

Quote:
“To explain, the ACTVTRC qmgr attribute is used where there is no
matching stanza in mqat.in for the app, or where no setting have been
overridden by values set in the MQCNO structure when the app connected
to the qmgr. The hierarchy is:

1. ACTVTRC qmgr attribute
overridden by
2. ACTVCONO settings (in MQCNO structure passed in MQCONNX)
overridden by
3. settings in a matching stanza in mqat.ini “


Additionally, the ACTVTRC property needs to be toggled to get the QM to read the mqat.ini file again to pick up any changes you make in the mqat.ini file.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
jcv
PostPosted: Wed Aug 20, 2014 8:02 am    Post subject: Re: Application Activity Trace - Keeps going and going.... Reply with quote

Chevalier

Joined: 07 May 2007
Posts: 411
Location: Zagreb

PeterPotkay wrote:
Anyone else seen this or any documentation that talks about it? It doesn’t seem ACTVTRC set to OFF means off!


I've seen and wrote about this here:

http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=44314

As you can see, I have also already requested enhancements with respect to selectivity of activity tracing, back in February 2014, and received 2033 from IBM Development team.
Back to top
View user's profile Send private message Visit poster's website
tczielke
PostPosted: Wed Aug 20, 2014 8:28 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

Hi jcv,

Did you paste the wrong link there? That link points to an RFE that is not about the Application Activity Trace (AAT).

If you are looking for a better tool for doing selections on the AAT, check out the amqsactz source on the Capitalware web site -> http://www.capitalware.com/mq_code_c.html

It produces 1 line summaries that include the RecordNum, Conn, Channel Name, Pid, and Tid. You can then grep on these results to do selections by those fields. Also, it produces a handy summary report with the -r option where you can see all the applications that were found in your AAT records, and what objects, channels, and operations the applications referenced. Finally, you could use the RecordNum form the 1 line summary info, to search back into a verbose report if you want more details on an API call.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Wed Aug 20, 2014 9:13 am    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

jcv, in your comments in that RFE I noticed you had issues with MS0P and MQExplorer locking up when looking at App Activity Trace messages.

So did I.

Turns out it was a bug in MQ that was causing the MQ channel process amqrmppa to die, which took out the channel used by MQ Explorer, and all the other channels in that amqrmppa process (up to 63 others).

This is the APAR produced by my PMR on this bug.
http://www-01.ibm.com/support/docview.wss?uid=swg1IT03124
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
jcv
PostPosted: Wed Aug 20, 2014 11:10 pm    Post subject: Reply with quote

Chevalier

Joined: 07 May 2007
Posts: 411
Location: Zagreb

Hi Tim and Peter, thanks for the information shared here. Although that RFE started with something different, as IBM pointed me towards AAT, I took the investigative path very similar to Peter's, observing need for additional possibilities of narrowing the trace, wondering what they did with respect to enabling/disabling the trace, how did they implement it and why, and finally observing the MQExplorer locking up when looking at App Activity Trace messages.
Back to top
View user's profile Send private message Visit poster's website
zpat
PostPosted: Thu Aug 21, 2014 12:43 am    Post subject: Reply with quote

Jedi Council

Joined: 19 May 2001
Posts: 5849
Location: UK

That PTF deserves to be Hiper in my view.

What a -n a s t y- bug - you're trying to solve a problem with a trace - and bingo you can take out other applications using the same client channel.

[changed by admin to not read contact admin bug...]
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error.
Back to top
View user's profile Send private message
tczielke
PostPosted: Thu Aug 21, 2014 9:22 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

Since we are talking about PMRs and the Application Activity Trace (AAT), one thing I have noticed at v8 is that the AAT seems to be missing some fields.

For example, when I compare the fields in a PUT between v7.5 and v8, I see the following fields missing in v8:

Message Data:
Object_type:
Object_name:
Object_Q_mgr_name:
Resolved_Q_Name:
Resolved_Q_mgr:
Resolved_local_Q_name:
Resolved_local_Q_mgr:
Resolved_type:

One of those missing fields that is important is the Object_name, as now in v8 you get a blank Object_name on your PUTs and GETs.

I have opened up a PMR for this issue, which is currently under investigation by IBM.

Unwittingly, I added functionality into the amqsactz program to get the object name from the OPEN call (the OPEN does still have the Object_name at v and then reinsert it into any GET, PUT or API call that has a valid HObj. I did this because I was doing testing of the amqsactz against v8 and thought this behavior of the missing Object_names in the GETs, PUTs, etc. was standard behavior. So I do have a version of amqsactz that "corrects" this issue of missing Object_names at v8, if anyone is struggling with this behavior at v8 and is interested in getting a fix to it from an amqsact stand point.
Back to top
View user's profile Send private message
Michael Dag
PostPosted: Mon Oct 27, 2014 4:36 am    Post subject: Reply with quote

Jedi Knight

Joined: 13 Jun 2002
Posts: 2602
Location: The Netherlands (Amsterdam)

anyone been able to stop logging of 'MQ Explorer 8.0.0' on Linux qmgrs in the mqat.ini ?

I tried ApplName=M* ApplName=MQ* but no luck...
in the trace records information from 'MQ Explorer 8.0.0' keeps coming up...
_________________
Michael



MQSystems Facebook page
Back to top
View user's profile Send private message Visit poster's website MSN Messenger
tczielke
PostPosted: Mon Oct 27, 2014 4:39 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

Did you toggle a queue manager attribute (i.e. ACTVTRC) after making the mqat.ini change? For running applications to pick up an mqat.ini change, you need to change a queue manager attribute to force the running application to read the mqat.ini file again and pick up the change. Even changing the decription attribute of your queue manager works.
Back to top
View user's profile Send private message
Michael Dag
PostPosted: Mon Oct 27, 2014 4:48 am    Post subject: Reply with quote

Jedi Knight

Joined: 13 Jun 2002
Posts: 2602
Location: The Netherlands (Amsterdam)

yes I did, several times with these options, off and on
_________________
Michael



MQSystems Facebook page
Back to top
View user's profile Send private message Visit poster's website MSN Messenger
tczielke
PostPosted: Mon Oct 27, 2014 4:53 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

Here are some usability notes on using the AAT:

Quote:
There is a hierarchy to turning ON/OFF the AAT:
1. ACTVTRC queue manager attribute (ON/OFF)
(overridden by)
2. MQCNO_ACTIVITY_TRACE connection options specified in an MQCONNX (NOTE: ACTVCONO queue manager attribute must be ENABLED for this to be checked, and the default value is DISABLED)
(overridden by)
3. Settings in a matching stanza in mqat.ini (located in qm.ini directory)

NOTE: In order to pick up a mqat.ini change dynamically in a running program, you need to change/toggle a queue manager attribute (i.e. ACTVTRC ON/OFF).


When the MQ Explorer 8.0.0 is writing out activity trace records, how do you have those 3 options mentioned above set? How does the application name of the MQ Explorer appear in the activity trace records?
Back to top
View user's profile Send private message
Michael Dag
PostPosted: Mon Oct 27, 2014 6:22 am    Post subject: Reply with quote

Jedi Knight

Joined: 13 Jun 2002
Posts: 2602
Location: The Netherlands (Amsterdam)

turn on/off tracing using explorer ACTVTRC ON/OFF/ON

ACTVCONO(ENABLED)

in mqat.ini
Code:
##################################################################
# Prevent the MQExplorer program from generating data            #
##################################################################
ApplicationTrace:           # Application specific settings stanza
  ApplClass=ALL             # Application type
                            #   Values:  (USER | MCA | ALL)
                            #   Default: USER
  ApplName=MQ*                  # Application name (may be wildcarded)
                            #   (matched to app name without path)
                            #   Default: *
  ApplFunction=*            # Application function (may be wildcarded)
                            #   (matched to app function)
                            #   Default: *
  Trace=OFF                 # Activity trace switch for application
                            #   Values:  ( ON | OFF )
                            #   Default: OFF
  ActivityInterval=0        # Time interval between trace messages
                            #   Values: 0-99999999 (0=off)
                            #   Default: 0
  ActivityCount=0           # Number of operations between trace msgs
                            #   Values: 0-99999999 (0=off)
                            #   Default: 0
  TraceLevel=MEDIUM         # Amount of data traced for each operation
                            #   Values: LOW | MEDIUM | HIGH
                            #   Default: MEDIUM
  TraceMessageData=0        # Amount of message data traced
                            #   Values: 0-104857600


resulting in following in amqsact output:

Code:
ApplicationName: 'MQ Explorer 8.0.0'
Application Type: MQAT_JAVA
ApplicationPid: 44322

_________________
Michael



MQSystems Facebook page
Back to top
View user's profile Send private message Visit poster's website MSN Messenger
Michael Dag
PostPosted: Mon Oct 27, 2014 6:24 am    Post subject: Reply with quote

Jedi Knight

Joined: 13 Jun 2002
Posts: 2602
Location: The Netherlands (Amsterdam)

on a Windows queue manager this works fine as the MQ Explorer application name is MQExplorer.exe ... no blank in the name...
_________________
Michael



MQSystems Facebook page
Back to top
View user's profile Send private message Visit poster's website MSN Messenger
tczielke
PostPosted: Mon Oct 27, 2014 6:27 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

In the amqsact output, what is it showing for the connection options for the MQ Explorer 8.0.0?
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » IBM MQ Performance Monitoring » Application Activity Trace - Keeps going and going....
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.