| Author |
Message
|
| mqxplorer |
 Posted: Wed Feb 06, 2013 12:50 pm Post subject: Auditing Queue messages |
 |
|
Master
Joined: 22 Jun 2009
Posts: 206
|
Hi All,
I am not sure if I am asking the right question. We have a situation where application A puts messages to queue 'Q' and application B reads from the same queue and these persistent messages. We are using circular logging. Ther are conflicts between application A and application B where people maintinaging application A say that they sent n number of messages and application B people say they received n-1 or some number you can pick. In order to find out the reality where we can actually see how many were put and how many were consumed from the queue, I thought we can use MQ logs. Based on my understanding about MQ logs I read in info center and some info posted by Peter Potkay in this forum:
| Code: |
| Persistent messages are written to the Linear and Circular logs inside of a unit of work. Once the message is committed, a circular log no longer has the message, but the linear does. |
we cannot use the logs for the auditing purposes. Is there a way we can monitor the queue and tell the counts (no of puts and gets per day or for a particluar time interval)? We use Tivoli monitoring for queues and our admins say we can enable the auidting for a particular duration like an hour or so and we can tell the count for that particular period only.
Please suggest me the possible options.
Thanks
mqxplorer |
|
| Back to top |
|
 |
| Vitor |
| Posted: Wed Feb 06, 2013 1:00 pm Post subject: Re: Auditing Queue messages |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqxplorer wrote: |
| Is there a way we can monitor the queue and tell the counts (no of puts and gets per day or for a particluar time interval)? |
Look up Monitoring and Statistics in the WMQ InfoCenter.
| mqxplorer wrote: |
| We use Tivoli monitoring for queues and our admins say we can enable the auidting for a particular duration like an hour or so and we can tell the count for that particular period only. |
If you already have Tivoli in place that's probably going to be the fastest easiest solution to obtain this information.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqxplorer |
| Posted: Wed Feb 06, 2013 1:19 pm Post subject: Re: Auditing Queue messages |
|
|
Master
Joined: 22 Jun 2009
Posts: 206
|
| Vitor wrote: |
If you already have Tivoli in place that's probably going to be the fastest easiest solution to obtain this information. |
We have Tivoli, however, our admins say that they can enable the auditing for a particular duation not like anytime. Effectively what I need is... no of puts and gets for the queue in question when the application teams ask. It should not be like time based. We have auit mechanism implemented for our flows and if consumer or provider application team asks for the count I just go to our audit table and give them based on the transaction start and end events configured for the nodes used in the flows. I need something similar to that on a queue where there is no message flow involved. Is that possible/ If so, can somebody suggest me the way to achieve it?
Thanks
mqxplorer |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Feb 06, 2013 1:46 pm Post subject: Re: Auditing Queue messages |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqxplorer wrote: |
| I need something similar to that on a queue where there is no message flow involved. Is that possible/ If so, can somebody suggest me the way to achieve it? |
| Vitor wrote: |
| Look up Monitoring and Statistics in the WMQ InfoCenter. |
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Feb 06, 2013 3:30 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| This sounds more like an MQ question than a broker question. You may get more useful responses from the MQ forum. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Feb 07, 2013 5:40 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| kimbert wrote: |
| This sounds more like an MQ question than a broker question. You may get more useful responses from the MQ forum. |
Good catch. Moved.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Feb 07, 2013 5:55 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
It's pretty hard to get this information accurately, and bear in mind that browsing a message is also a MQGET.
What I did in a similar situation is set up an automatic duplication of the messages (using a topic alias and two subscriptions).
The second copy of the messages then allowed the originators to demonstrate that they had produced the ones they said they had. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Feb 07, 2013 6:06 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
The accounting and statistics should allow you to see exactly how many messages were put and exactly how many messages were got.
If the number of messages put is different than team A says, then that tells you something.
If the number of messages got is different than team B says, then that tells you something.
if the number of messages put and got are the same, then no messages were "lost".
If the number of messages put and got are the same as each team says, then that tells you that they merely disagree on the interval of service. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Feb 07, 2013 8:11 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Ah yes, that easy to use feature "accounting and statistics"....
Bear in mind changes to these settings only take effect when the queue is next opened.
It stands to reason that an empty queue, has had as many destructive gets as puts done to it. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Feb 07, 2013 8:19 am Post subject: Re: Auditing Queue messages |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| mqxplorer wrote: |
Hi All,
I am not sure if I am asking the right question. |
Are these new or newly modified applications?
Have they ever worked correctly?
Do they fail all of the time? Or just occasionally?
Is the failure repeatable?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Feb 07, 2013 8:23 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Feb 07, 2013 9:03 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| zpat wrote: |
| It stands to reason that an empty queue, has had as many destructive gets as puts done to it. |
Apps that rollback messages only to pick them up again will drive the # of gets higher than the number of puts.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mqxplorer |
| Posted: Thu Feb 07, 2013 1:03 pm Post subject: |
|
|
Master
Joined: 22 Jun 2009
Posts: 206
|
| mqjeff wrote: |
The accounting and statistics should allow you to see exactly how many messages were put and exactly how many messages were got.
If the number of messages put is different than team A says, then that tells you something.
If the number of messages got is different than team B says, then that tells you something.
if the number of messages put and got are the same, then no messages were "lost".
If the number of messages put and got are the same as each team says, then that tells you that they merely disagree on the interval of service. |
As suggested by in another thread http://www.mqseries.net/phpBB2/viewtopic.php?t=44891&highlight=
I downlaoed MS0T and tried to look for the count for no of puts and gets and it is bit confusing.. may be the document I read is not that good or I did not understand it right.
I was playing around with the explorer options to get the stats and I observed something which I feel is not a good feature or at least not what I am looking for.
I created a queue and enabled the accounting feature ACCTQ(ON) and connected the QM and put messages on the queue using RFHUtil. I thought I would see the no of puts and gets on this queue when I refresh the explorer's Events and Statistics feature. However, I could not see any for this queue under the application: rfhutilc.exe and when I closed the rfjutil and refreshed the Events and Statistics window, it displays the updated puts, gets, browse etc. If the tool cannot update the stats if the application is still connected the queue manager, then it will not be helpful as we cannot ask the application B reading messages from queue Q to dicsoneect so that we can tell the exact count as it PROD environment.
Is my observatin correct or I am missing something?
Thanks
mqxplorer |
|
| Back to top |
|
|
| mqxplorer |
| Posted: Thu Feb 07, 2013 1:15 pm Post subject: Re: Auditing Queue messages |
|
|
Master
Joined: 22 Jun 2009
Posts: 206
|
| bruce2359 wrote: |
| mqxplorer wrote: |
Hi All,
I am not sure if I am asking the right question. |
Are these new or newly modified applications?
Have they ever worked correctly?
Do they fail all of the time? Or just occasionally?
Is the failure repeatable? |
This is a new application. The problem here is due to some people who work in IT dept who should have been in senate or some political offcie where they do not normally work and let anybody work in a positive way. Application A puts messages on queue Q, it says it published 100 messages and application B consumes the messages and says it consumed only 88 messages, hence the blame game starts. In one of my experiences with application B is they do not have a corret exception handling where if they get an invalid message from the queue or if there is an exception in processing one of the consumed messages, they neither write that message to any file as an exception case nor roll it back to queue. We experimented once by putting a poison message in the bunch of correct messages in DEV and asked them to comsume and they were telling that they consumed only 8 messages even though we put 9 messages. They do not have any clue what happened with that posised message. After having a lot of useless/non-directional meetings, we have decided to find out a solution to tell the count of messages on the queue (put and get) so that we will be clean from middleware point of view and the application can continue do what they want to do with their selfish politics...Hence I am in search of a tool which can make these guys shut up and do some real work...
Thanks
mqxplorer |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Feb 07, 2013 1:32 pm Post subject: Re: Auditing Queue messages |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| mqxplorer wrote: |
| ...we have decided to find out a solution to tell the count of messages on the queue (put and get) so that we will be clean from middleware point of view and the application can continue do what they want to do with their selfish politics... |
You have already demonstrated that the consumer application code is defective. Clearly, the developers are violating application best-practices.
It is a best-practice for an application to count the number of messages consumed from the queue, the number of messages successfully processed, and the number of messages unsuccessfully processed. It is a best-practice to do something with messages the app cannot successfully process.
You need to escalate this to management, and include your internal/external auditors in this, before your organization finds that it has lost millions of dollars due to bad application coding.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
|
|
