ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » IBM MQ Performance Monitoring » MQ9 monitoring bug?

Post new topic  Reply to topic
 MQ9 monitoring bug? « View previous topic :: View next topic » 
Author Message
fjb_saper
PostPosted: Sat Jun 18, 2016 10:18 pm    Post subject: MQ9 monitoring bug? Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

Environment is Windows 10 Home
Sequence of events:
  1. Create qmgr MQ9 in version 9
  2. start it
  3. create qlocal named MONITORING
  4. create subscription MONITOR in MQExplorer
    TOPICSTR('$SYS/MQ/INFO/QMGR/MQ9/Monitor/CPU/QMgrSummary')
    DEST(MONITORING)
  5. stop the qmgr with wait (-w)
  6. start the qmgr experience the crash of amqzmuc0

in the qmgr log it crashes while starting the PUB/SUB phase.

Anybody else hit the same problem?
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
hughson
PostPosted: Sun Jun 19, 2016 2:43 pm    Post subject: Reply with quote

Padawan

Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand

Just tried it out, and yes.

Upon start up:-

Code:
IBM MQ queue manager 'MQ900' starting.
The queue manager is associated with installation '900GA'.
5 log records accessed on queue manager 'MQ900' during the log replay phase.
Log replay for queue manager 'MQ900' complete.
Transaction manager state recovered for queue manager 'MQ900'.
AMQ6110: An internal IBM MQ error has occurred.


and in the AMQERR01.LOG:-
Code:
-------------------------------------------------------------------------------
20/06/2016 10:31:50 - Process(11004.1) User(MUSR_MQADMIN6) Program(amqzxma0.exe)
                      Host(PAMLAP) Installation(900GA)
                      VRMF(9.0.0.0) QMgr(MQ900)
                     
AMQ5008: An essential IBM MQ process 9604 (zllCRIT) cannot be found and is
assumed to be terminated.

EXPLANATION:
1) A user has inadvertently terminated the process. 2) The system is low on
resources.  Some operating systems terminate processes to free resources.  If
your system is low on resources, it is possible it has terminated the process
so that a new process can be created.
ACTION:
IBM MQ will stop all MQ processes.  Inform your systems administrator.  When
the problem is rectified IBM MQ can be restarted.
----- amqzxmb0.c : 10085 ------------------------------------------------------


and an FDC (snipped a little):-
Code:

+-----------------------------------------------------------------------------+
|                                                                             |
| IBM MQ First Failure Symptom Report                                         |
| =========================================                                   |
|                                                                             |
| Probe Id          :- ZD040010                                               |
| Application Name  :- MQM                                                    |
| Component         :- zruNotify                                              |
| SCCS Info         :- F:\build\slot1\p900_P\src\lib\zu\amqzruaa.c,           |
| Line Number       :- 963                                                    |
| Build Date        :- May 20 2016                                            |
| Build Level       :- p900-L160520.DE                                        |
| Build Type        :- IKAP - (Production)                                    |
| Process Name      :- C:\mqm9000\bin64\amqzmuc0.exe                          |
| Arguments         :- -m MQ900                                               |
| Addressing mode   :- 64-bit                                                 |
| Process           :- 00009604                                               |
| Thread            :- 00000018    DurSubsMgr                                 |
| Session           :- 00000000                                               |
| QueueManager      :- MQ900                                                  |
| UserApp           :- FALSE                                                  |
| ConnId(1) IPCC    :- 23                                                     |
| ConnId(2) QM      :- 39                                                     |
| Last HQC          :- 2.6.6-196800                                           |
| Last HSHMEMB      :- 2.6.6-667840                                           |
| Last ObjectName   :-                                                        |
| Major Errorcode   :- xecS_E_MEM_SET_NOT_FOUND                               |
| Minor Errorcode   :- OK                                                     |
| Probe Type        :- INCORROUT                                              |
| Probe Severity    :- 2                                                      |
| Probe Description :- AMQ6125: An internal IBM MQ error has occurred.        |
| FDCSequenceNumber :- 0                                                      |
|                                                                             |
+-----------------------------------------------------------------------------+

MQM Function Stack
zmuThreadMain
zmuDurableSubscriptionManagerTask
kpiDurableSubscriptionManagerTask
kqiReconcileDurableSubscribers
kqiTopicRegisterSubscriber
zruNotify
xcsFFST


Looks like the code which processes the durable subscriptions at start up has had a problem. I had a play with this before, but I think I removed the subscriptions again before I restarted the queue manager so I never had the position to see this before.

Cheers
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software
Back to top
View user's profile Send private message Visit poster's website
Andyh
PostPosted: Sun Jun 19, 2016 11:31 pm    Post subject: Reply with quote

Master

Joined: 29 Jul 2010
Posts: 237

It looks like there's a race condition during startup.
This works fine in my V9 environment, but looking at the code it's clear there's no explicit serialization to cover this case.
Please raise a PMR in order that this issue can be resolved (via APAR).
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Mon Jun 20, 2016 4:46 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

Andyh wrote:
It looks like there's a race condition during startup.
This works fine in my V9 environment, but looking at the code it's clear there's no explicit serialization to cover this case.
Please raise a PMR in order that this issue can be resolved (via APAR).

No PMRs for the developer edition...
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
Andyh
PostPosted: Mon Jun 20, 2016 8:20 am    Post subject: Reply with quote

Master

Joined: 29 Jul 2010
Posts: 237

The fix has been written/tested in Hursley, but I have no way of getting it to you other than via an APAR.

As a workaround you might consider remaking a non-durable subscription each time the queue manager starts.

Conceptually, the metadata is related to a particular queue manager session. For example, if you applied service (or even changed the configuration) then the queue manager would be at liberty to start publishing different monitoring data (which would be described by the metadata associated with that queue manager session) and hence a non-durable subscription might seem a more typical choice.

We wanted to keep the published data as dynamic as possible in order that it could reflect MQ internals, where earlier stats reflected only MQ externals. For example we now publish some measure of I/O latency , queue lock contention, and queue avoidance. In future releases of MQ we would feel free to change these stats to reflect the internals of the current MQ implementation, and hence would prefer that subscriptions were similarly dynamic (e.g non-durable) and based upon the 'currently' published metadata.

Have you reviewed the amqsruaa.c sample that demonstrates how to consume the metadata and then make appropriate subscriptions.
The web based GUI for the MQ appliance is also an example (not source!!!) of how the metadata can be consumed and the underlying monitoring publications displayed.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Mon Jun 20, 2016 9:30 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

I did look at amqsruaa...

I was thinking here of requesting the metadata in a lazy way, i.e. on demand when I consume a monitoring message. This way the subscription can be durable and when the consumption program starts up it will acquire the relevant metadata upon parsing of the pcf message.... Obviously the metadata would then be cached until the consumer program is done...

By the way is there a way to make the subscriptions accept a wild card?

Monitoring topics at this point do not accept a wildcard in them...
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
Andyh
PostPosted: Mon Jun 20, 2016 12:03 pm    Post subject: Reply with quote

Master

Joined: 29 Jul 2010
Posts: 237

The metadata include the topic strings upon which the relevant data is published and so it seems a little bit chicken and egg to use the topic string to find the metadata rather than the other way around.
Similarly, given that we've published the set of topic strings that can sensibly be subscribed to, why would you want to make a wildcard subscription.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Mon Jun 20, 2016 2:20 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

Andyh wrote:
The metadata include the topic strings upon which the relevant data is published and so it seems a little bit chicken and egg to use the topic string to find the metadata rather than the other way around.
Similarly, given that we've published the set of topic strings that can sensibly be subscribed to, why would you want to make a wildcard subscription?


Because I might want to subscribe to all possible topics for a queue manager instead of having to do half a dozen of subscriptions or more...
example would be:
$SYS/MQ/INFO/QMGR/#
or
$SYS/MQ/INFO/QMGR/+/Monitor/DISK/#


_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
Andyh
PostPosted: Tue Jun 21, 2016 1:13 am    Post subject: Reply with quote

Master

Joined: 29 Jul 2010
Posts: 237

This has a little bit of a feel of "giving you enough rope to hang yourself".

At this point in time I think I'm more comfortable with the idea of both more transient and more specific subscriptions to the new MQ monitoring topics.

Once this facility has matured a little then perhaps wildcard subscirptions should be enabled at some level, but in the short term I think it's quite prudent to introduce this capability in a more restrictive manner.

As already explained, the metadata can be used to dynamically determine what monitoring data is supported by the current queue manager (which varies a little by platform) and then specific subscriptions made for the data that you want to monitor.
One of the primary goals here is to keep this stuff as dynamic and flexible as possible, and that seems somewhat dependent upon customers/vendors making full/proper use of the metadata.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » IBM MQ Performance Monitoring » MQ9 monitoring bug?
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.