| Author |
Message
|
| sg94k6r2 |
 Posted: Fri Sep 03, 2004 11:24 am Post subject: Maximum Queue Depth |
 |
|
Newbie
Joined: 03 Sep 2004
Posts: 8
|
Hello,
We have a a program that puts trace messages on the queue. This queue can get filled up quickly because the messages do not go anywhere but just sit there for auditing and tracking purpose. We would like to clear the queues out once a week and preferably archive the messages in a file somewhere.
We have MQ 5.3 on Windows 2000. Do we have to write a code and schedule it? Can we do it using some built in MQ Funcationality? If we have to write a code a sample would be much appreciated.
Thanks
AZXML |
|
| Back to top |
|
 |
| PeterPotkay |
| Posted: Fri Sep 03, 2004 11:50 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Put the messages to the queue with 7 days expiry.
Schedule a job to run every day that starts up an MQ program that does one MQGET specifying a non existsent Correl ID. The MQGET will not pull off any messages, but in the process of looking for a message with that bogus Correl ID, it will touch every message in the queue, discarding the expired ones.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| bbburson |
| Posted: Fri Sep 03, 2004 11:55 am Post subject: |
|
|
Partisan
Joined: 06 Jan 2004
Posts: 378
Location: Nowhere near a queue manager
|
| PeterPotkay wrote: |
Put the messages to the queue with 7 days expiry.
Schedule a job to run every day that starts up an MQ program that does one MQGET specifying a non existsent Correl ID. The MQGET will not pull off any messages, but in the process of looking for a message with that bogus Correl ID, it will touch every message in the queue, discarding the expired ones. |
...but that won't satisfy this:
| sg94k6r2 wrote: |
and preferably archive the messages in a file somewhere.
|
|
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Sep 03, 2004 12:19 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
oops, missed that little detail. If you want to archive them, then my method is no good.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Sep 03, 2004 7:26 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Set up High queue event.
Monitor the event queue and when this high event happens
Have a process read the messages in the queue and put them into a file.
The Message class should be Serializable so you can just write the object to the file.
Make sure you flush before closing...
This should avoid the queue overflowing and will take into account that your queue activity is not steady but comes most probably in spurts...
Enjoy
F.J. |
|
| Back to top |
|
|
| kman |
| Posted: Mon Sep 06, 2004 12:22 am Post subject: |
|
|
Partisan
Joined: 21 Jan 2003
Posts: 309
Location: Kuala Lumpur, Malaysia
|
| Quote: |
| We would like to clear the queues out once a week and preferably archive the messages in a file somewhere. |
| Quote: |
Do we have to write a code and schedule it? Can we do it using some built in MQ Funcationality? If we have to write a code a sample would be much appreciated.
|
The sample amqsget can do this quickly. Edit the program to take in bigger message buffer (as big as your message length). Then write the message to a file. Run this program every week. 
Or look for MQ VisualEdit program, - run a search on this site for tht. Or download the RFHUtil from the supportpac site. |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Mon Sep 06, 2004 10:22 am Post subject: |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3264
Location: London, ON Canada
|
Hi,
Yes, MQ Visual Edit will do what you needm but it is a GUI program that means you cannot run it as a server script.
For more information or to download a 30-day trial of MQ Visual Edit, go to:
http://www.capitalware.biz/products.html
If you can wait less then a week, I might have a product that you are looking for. For the past couple of months, I been creating a new product based on MQ Visual Edit but no GUI. Everything runs from a command prompt or shell scripts. It has all 23 major features of MQ Visual Edit but runs quiet and uses Log4J to hold the results.
To do what you require would take one line that you could schedule by crontab (the '-D' means Destructive Get ):
| Code: |
| mqbt.sh -p MQA1 -q TEST01.Q -f d:\mybackup.veq -D |
It will be called MQ Batch Toolkit and the coding and documentation are done. I am currently updating my web site and finishing the language translation. I'm hoping to release it next weekend.
Regards,
Roger Lacroix
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| Nigelg |
| Posted: Wed Sep 08, 2004 1:33 am Post subject: |
|
|
Grand Master
Joined: 02 Aug 2004
Posts: 1046
|
| Peter Potkay's reply is wrong. Expired msgs are only removed from the queue if they would otherwise have been returned to the caller. In the case where a bogus CorrelID is passed into MQGET, the expired msg would not match and so would not be removed. A good solution would be to use amqsbcg to browse the entire queue removing expired msgs as it goes. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Sep 08, 2004 4:06 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
No, my answer is correct.
As MQ "touches" each message to see if the Correl ID is a match, the message is discarded if it is expired. Any messages that are expired that were not touched, i.e. they are in the queue below your retrieved message, are not discarded. That is why searching for a bogus Correl ID will delete any expired messages on the queue, since it will look at the queue top to bottom.
Test it yourself and see.
If this were not the case, all our queues everywhere would have problems of bogus depths as expired messages accumalated. Instead, as the queue is used by apps doing a get by Correl ID for their reply, they constantly flush the queue of any expired messages that may have been there ahead of their reply.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Nigelg |
| Posted: Wed Sep 08, 2004 5:12 am Post subject: |
|
|
Grand Master
Joined: 02 Aug 2004
Posts: 1046
|
Check out the manual:
.
The manual (APR, MQMD, Expiry) states:
.
After a message's expiry time has elapsed, it becomes eligible to be
discarded by the queue manager. In the current implementations, the
message is discarded when a browse or nonbrowse MQGET call occurs
that would have returned the message had it not already expired. For example, a nonbrowse MQGET call with the MatchOptions field in MQGMO set to MQMO_NONE reading from a FIFO ordered queue will cause all the expired messages to be discarded up to the first unexpired message.
.
See the bold bit. If match by CorrelId is done with the bogus CorrelId, and the msg on the queue does not match the CorrelId passed in, the msg will not be discarded because it would not have been returned had it not already expired. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Sep 08, 2004 5:25 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Test it yourself and see.
Put the following messages to a queue:
Message # CorrelID Expiry
1 abc1 10
2 abc2 10
3 abc3 10
4 PETER 1000000
5 abc4 10
6 abc5 10
Wait a minute. The queue now has 6 messages, five of which are expired. Queue depth is 6.
Do an MQGET for Correl ID PETER. Messages 1,2 and 3 will be discarded, Message 4 will be returned, Message 5 and 6 will be left. Queue depth after the test wil be 2 (messages 5 and 6, both expired).
Do the test again, but this time search for Correl ID NIGELG. This time, all the message will be touched, you will get a 2033, and the queue depth will be 1.
I know what the manual says, but try it for real.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Nigelg |
| Posted: Wed Sep 08, 2004 5:53 am Post subject: |
|
|
Grand Master
Joined: 02 Aug 2004
Posts: 1046
|
I have tried this, and it does not remove the expired msgs if the CorrelId does not match.
I created 6 msgs with a CorrelId created by the system, and an Expiry of 10 = 1 second.
I then waited 2 seconds, and did an MQGET (modified amqsget) specifying CorrelId "PETER", i.e. strncpy(md.CorrelId,"PETER",5) just before the MQGET.
A DIS QL(..) CURDEPTH then showed 6.
I then ran amqsbcg on the same queue, and the CURDEPTH was 0.
Interesting... |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Sep 08, 2004 5:57 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Hmmm, let me try it as well. Its been a couple of years since I did this test. What platform / version are you on?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Nigelg |
| Posted: Wed Sep 08, 2004 6:02 am Post subject: |
|
|
Grand Master
Joined: 02 Aug 2004
Posts: 1046
|
I have the answer.
When I modifed amqsget0.c, I did not uncomment the lines:
/* gmo.Version = MQGMO_VERSION_2; */
/* gmo.MatchOptions = MQMO_NONE; */
When I did do this, the behaviour is as Peter described.
When I used a GMO V2, I had to change the gmo.MatchOptions to MQMO_MATCH_CORREL_ID to get the previous default behaviour at GMO V1.
So, we are both right!
Expired msgs are only removed if they would otherwise be returned.
If the GMO version is 2, then the MatchOptions have to be set explicitly to MQMO_MATCH_CORREL_ID to check the CorrelId.
If the GMO version is 1, then the MatchOptions do not have to be set for the CorrelId in the MD to be checked when getting a candidate msg for retrieval and by extension for deletion of an expired msg. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Sep 08, 2004 6:10 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| Quote: |
If the GMO version is 2, then the MatchOptions have to be set explicitly to MQMO_MATCH_CORREL_ID to check the CorrelId.
|
Are you sure? It is my understanding that if there is anything at all in the Message ID or Correl ID field, MQ will try and match on it, regardless of whether you have the Match Option set, or what version of GMO you are using. It has always been pounded into my head to always init these fileds otherwise MQ will try and match on them.
Can you try your test again, leaving the following line commented out, and trying it with gmo V1 and V2?
/* gmo.MatchOptions = MQMO_NONE; */
I think its what you have in the CorrelID field that counts really. Please try the test exactly as I have described it above (with expired message above and below the intended message).
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
|
|
