ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » General IBM MQ Support » Queue Statistics PUT & GET Count doesn't match

Post new topic  Reply to topic Goto page 1, 2  Next
 Queue Statistics PUT & GET Count doesn't match « View previous topic :: View next topic » 
Author Message
ibmmqrock
PostPosted: Fri Sep 13, 2019 12:49 pm    Post subject: Queue Statistics PUT & GET Count doesn't match Reply with quote

Novice

Joined: 30 May 2019
Posts: 14

Hi All, I just ran a 2 hour statistics on our local Reply Q using the "ALTER QLOCAL STATQ(on) command.
and now I am checking the statistical data and the GET count is higher than PUT count, which is confusing as both should match , in some cases GET count could be lower if there are messages which has EXPIRED or PURGED, but how come GET Count is higher.

Quote:
QueueType: Local
QueueDefinitionType: Predefined
QMinDepth: 0
QMaxDepth: 1
AverageQueueTime: [0, 2121]
PutCount: [0, 897]
PutFailCount: 0
Put1Count: [0, 0]
Put1FailCount: 0
PutBytes: [0, 2789307]
GetCount: [0, 1308]
GetBytes: [0, 4838004]
GetFailCount: 3
BrowseCount: [0, 0]
BrowseBytes: [0, 0]
BrowseFailCount: 0
NonQueuedMsgCount: 0
ExpiredMsgCount: 0
PurgeCount: 0
Any thoughts?
Back to top
View user's profile Send private message
bruce2359
PostPosted: Fri Sep 13, 2019 3:26 pm    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

Firstly, is your app missing messages?

I'd expect to see get count higher than put count if the app "polls" the queue looking for messages, but a r/c of 2033 NO_MSG_AVAILABLE is returned with either MQCC_FAILED or MQCC_WARNING

Did you notice the GetFailCount: 3? Did your app log anything about these?
How did the app recover from these failures? Take a look at the MQ development manual (or KC) for MQGET reason codes (associated with MQCC_FAILED). Possibilities include queue get inhibited, message too big for the app buffer, and dozens of others.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.


Last edited by bruce2359 on Sat Sep 14, 2019 7:10 am; edited 3 times in total
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Fri Sep 13, 2019 7:58 pm    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

MQRC 2033 does not increase the get count. Nor do failed gets.

More gets than puts is because the app is rolling back its transactions, backing out messages onto the queue and picking them up again. The rollback does not count as another put. Picking up the same message again drives the get count up.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
bruce2359
PostPosted: Sat Sep 14, 2019 4:49 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

From https://www.ibm.com/developerworks/community/blogs/messaging/entry/Statistics_published_to_the_system_topic_in_MQ_v9?lang=en
Quote:
Statistics of failed MQI calls are interesting because they enable customers to troubleshoot poorly-written applications that generate unnecessary failed MQI calls, thereby impacting performance.

Some examples of failing reasons for various MQI calls that would be recorded in the statistics are...

- when MQCONN/MQCONNX/MQOPEN returns 2035 MQRC_NOT_AUTHORIZED when diagnosed by the queue manager, not the client. For example running amqsput as nobody.
- when MQPUT/MQPUT1 returns 2053 MQRC_Q_FULL because MAXDEPTH has been exceeded.
- when MQGET returns 2033 MQRC_NO_MSG_AVAILABLE when browsing or destructively getting from an empty queue
- when MQSUBRQ returns 2437 MQRC_NO_RETAINED_MSG because there is no retained message

_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
ibmmqrock
PostPosted: Sat Sep 14, 2019 12:10 pm    Post subject: Reply with quote

Novice

Joined: 30 May 2019
Posts: 14

PeterPotkay wrote:
MQRC 2033 does not increase the get count. Nor do failed gets.

Yes, that is my understanding too.
PeterPotkay wrote:

More gets than puts is because the app is rolling back its transactions, backing out messages onto the queue and picking them up again. The rollback does not count as another put. Picking up the same message again drives the get count up.


It might be the case, that the application is rolling back its transactions, although hard to figure that out as Queue Statistics doesn't return count of rollback messages.
Back to top
View user's profile Send private message
Andyh
PostPosted: Sun Sep 15, 2019 9:58 pm    Post subject: Reply with quote

Master

Joined: 29 Jul 2010
Posts: 237

The most obvious reason for this mismatch would seem to be that the queue wasn't empty at the start of the stats interval.
amqsrua might give you some additional insight here, that program reports on a 10 second cadence, but MQ doesn't want a 10 second delay before anything is reported and so the initial publication reports the current value of the counters (rather than the delta's since the last report). This is more complicated if there are multiple subscriptions to these stats, but it might just give you your answer.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Mon Sep 16, 2019 9:04 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

PeterPotkay wrote:
MQRC 2033 does not increase the get count. Nor do failed gets.

Uh...

MQCC_FAILED is returned with MQRC_NO_MSG_AVAILABLE

The OP posted:
Quote:
GetCount: [0, 1308]
GetBytes: [0, 4838004]
GetFailCount: 3


How is it that neither GetCount nor GetFailed count are incremented for 2033's?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Sep 19, 2019 9:51 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

PeterPotkay wrote:
MQRC 2033 does not increase the get count. Nor do failed gets.

I can find no reference to r/c 2033's not being counted in queue statistics. Can you offer a citation? Or, a rationale why 2033's aren't counted as MQGET failures?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Thu Sep 19, 2019 2:14 pm    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

Neither can I. A simple test will prove it though.

As for rationale? "GetCount". One could infer it means how many messages were gotten from the queue. Its not unfair for someone else to infer it means # of MQGET calls issued, regardless of results.

I could see value in both metrics being (clearly) provided as 2 separate metrics. But as of now, that particular metric does not include failed MQGETs, not even MQRC 2033s.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
hughson
PostPosted: Thu Sep 19, 2019 2:21 pm    Post subject: Reply with quote

Padawan

Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand

bruce2359 wrote:
PeterPotkay wrote:
MQRC 2033 does not increase the get count. Nor do failed gets.

I can find no reference to r/c 2033's not being counted in queue statistics. Can you offer a citation? Or, a rationale why 2033's aren't counted as MQGET failures?

2033 and other failures of MQGET calls do not increase the GetCount, they increase the GetFailedCount. I don't believe Peter said they are not counted as get failures.

Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software
Back to top
View user's profile Send private message Visit poster's website
HubertKleinmanns
PostPosted: Thu Sep 19, 2019 11:13 pm    Post subject: Reply with quote

Shaman

Joined: 24 Feb 2004
Posts: 732
Location: Germany

hughson wrote:
bruce2359 wrote:
PeterPotkay wrote:
MQRC 2033 does not increase the get count. Nor do failed gets.

I can find no reference to r/c 2033's not being counted in queue statistics. Can you offer a citation? Or, a rationale why 2033's aren't counted as MQGET failures?

2033 and other failures of MQGET calls do not increase the GetCount, they increase the GetFailedCount. I don't believe Peter said they are not counted as get failures.

Cheers,
Morag


I've tested it on a test QMgr: 2033's increase the GetFailCount, not the GetCount.

But this is not an answer to the question, why there are so many GetCounts. Could be, that the Queue wasn't empty before starting the test, but in this case I would expect a much higher value for AverageQueueTime.

I'm also interessted in an answer to question, why the GetCount is so much higher than PutCount?

Maybe there was a GET with syncpoint followed by a BACKOUT? How does this increase the statistics counter?
_________________
Regards
Hubert
Back to top
View user's profile Send private message Visit poster's website
hughson
PostPosted: Fri Sep 20, 2019 1:12 am    Post subject: Reply with quote

Padawan

Joined: 09 May 2013
Posts: 1914
Location: Bay of Plenty, New Zealand

HubertKleinmanns wrote:
hughson wrote:
bruce2359 wrote:
PeterPotkay wrote:
MQRC 2033 does not increase the get count. Nor do failed gets.

I can find no reference to r/c 2033's not being counted in queue statistics. Can you offer a citation? Or, a rationale why 2033's aren't counted as MQGET failures?

2033 and other failures of MQGET calls do not increase the GetCount, they increase the GetFailedCount. I don't believe Peter said they are not counted as get failures.

Cheers,
Morag


I've tested it on a test QMgr: 2033's increase the GetFailCount, not the GetCount.

Happy to hear we agree. Hopefully Bruce is happy now too, I was clarifying for him as he seemed to have mis-interpreted Peter's comment.

HubertKleinmanns wrote:
But this is not an answer to the question, why there are so many GetCounts. Could be, that the Queue wasn't empty before starting the test, but in this case I would expect a much higher value for AverageQueueTime.

I'm also interessted in an answer to question, why the GetCount is so much higher than PutCount?


There have been many possible suggestions. With only the statistics from after the fact, all the suggestions would be a guess though.

HubertKleinmanns wrote:
Maybe there was a GET with syncpoint followed by a BACKOUT? How does this increase the statistics counter?


This was indeed discussed earlier in this thread.

Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software
Back to top
View user's profile Send private message Visit poster's website
HubertKleinmanns
PostPosted: Fri Sep 20, 2019 2:11 am    Post subject: Reply with quote

Shaman

Joined: 24 Feb 2004
Posts: 732
Location: Germany

hughson wrote:
HubertKleinmanns wrote:
hughson wrote:
bruce2359 wrote:
PeterPotkay wrote:
MQRC 2033 does not increase the get count. Nor do failed gets.

I can find no reference to r/c 2033's not being counted in queue statistics. Can you offer a citation? Or, a rationale why 2033's aren't counted as MQGET failures?

2033 and other failures of MQGET calls do not increase the GetCount, they increase the GetFailedCount. I don't believe Peter said they are not counted as get failures.

Cheers,
Morag


I've tested it on a test QMgr: 2033's increase the GetFailCount, not the GetCount.

Happy to hear we agree. Hopefully Bruce is happy now too, I was clarifying for him as he seemed to have mis-interpreted Peter's comment.


I fear, I misunderstood Peter's remark too. I read it like "MQRC 2033 does not increase ... failed gets (Counter)." but I should read it like "... Nor do failed gets (increase the Get Counter).", right?


hughson wrote:
HubertKleinmanns wrote:
Maybe there was a GET with syncpoint followed by a BACKOUT? How does this increase the statistics counter?


This was indeed discussed earlier in this thread.

Cheers,
Morag


Sorry, I missed this contribution from Peter .

The Knowledge Center says
Quote:
The number of successful destructive get requests for persistent and nonpersistent messages. This parameter is an integer list indexed by persistence value.


So the new question is: Are read and backed out messages successful destructive get requests?
_________________
Regards
Hubert
Back to top
View user's profile Send private message Visit poster's website
Andyh
PostPosted: Fri Sep 20, 2019 2:46 am    Post subject: Reply with quote

Master

Joined: 29 Jul 2010
Posts: 237

Q. Are read and backed out messages successful destructive get requests?
A. Yes, the destructive get operation succeeded
Back to top
View user's profile Send private message
bruce2359
PostPosted: Fri Sep 20, 2019 4:45 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

HubertKleinmanns wrote:
I fear, I misunderstood Peter's remark too. I read it like "MQRC 2033 does not increase ... failed gets (Counter)." but I should read it like "... Nor do failed gets (increase the Get Counter).", right?

Yep, that was how I misread Peter's reply. I read it as 2033's are not counted as get fails. What a wild ride down that rabbit hole.

MQI stats would be helpful here. Also a concurrent MQI trace would be helpful.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » General IBM MQ Support » Queue Statistics PUT & GET Count doesn't match
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.