| Author |
Message
|
| klamerus |
 Posted: Tue May 10, 2005 6:00 pm Post subject: Expiring queues??? |
 |
|
Disciple
Joined: 05 Jul 2004
Posts: 199
Location: Detroit, MI
|
We have a system where messages for work are received over an MQ queue and quickly sent on to a variety of programs for handling.
Occasionally, these applications "screw up", and so we thought we'd place a copy of each message on what we call a backup queue. That we we always have a spare copy of the original message to do something with if a message gets "lost" in the works.
We only want to keep these for a week. If we haven't discovered the issue at that point, it's probably too late.
So, we thought we configured a queue that would "expire" messages that had reach > 1 week of age, but something's not right. For instance, the backup queues has > 250,000 messages on it even though our tracking database of completed requests shows only about 200,000 messages handled since beginning of month (which is > 1 week ago).
So we clearly haven't configured something right. Even more strange, it seems like maybe the oldest message is only in this range, but the queues are clearly not really being emptied.
Can someone tell us specifically something we could look for or check on to accomplish this behavior. We're clearly not doing something right. |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Tue May 10, 2005 6:15 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
What is your environment?
On zOS there is a program to remove expired messages and you would need to run it every so often.
On distributed services you need to browse or consume the queue in order to expire messages.
However I would question the design. What do you mean with messages getting lost? If a message gets lost in the works fix the application. Not the MQ system !!!
a) ensure you set all messages to persistent.
b) make sure all are handled under syncpoint
c) make sure you know how to handle a "poison message"
d) result..... no more lost messages and only the ones you can't process are kept (backout queue etc...)
Some reading up on the programming manual is in order !!
Enjoy  |
|
| Back to top |
|
|
| EddieA |
| Posted: Tue May 10, 2005 6:55 pm Post subject: |
|
|
Jedi
Joined: 28 Jun 2001
Posts: 2453
Location: Los Angeles
|
| Quote: |
| we configured a queue that would "expire" messages that had reach > 1 week of age |
A "queue" doesn't have an expiry, a message does. So, when you put the message on the queue you have to make sure that you have set the expiry correctly.
Also, after a message has expired, it will only be discarded when an application tries to read/browse the message, as pointed out by fjb_saper.
Cheers,
_________________
Eddie Atherton
IBM Certified Solution Developer - WebSphere Message Broker V6.1
IBM Certified Solution Developer - WebSphere Message Broker V7.0 |
|
| Back to top |
|
|
| bower5932 |
| Posted: Tue May 10, 2005 7:14 pm Post subject: |
|
|
Jedi Knight
Joined: 27 Aug 2001
Posts: 3023
Location: Dallas, TX, USA
|
| You ought to be able to browse the queue and see what the messages are and what there expiry is. I would think if you saw them it might explain a lot of things. If the browse actually gets less messages than are on the queue, then the expired ones were thrown away. |
|
| Back to top |
|
|
| klamerus |
| Posted: Wed May 11, 2005 3:40 pm Post subject: |
|
|
Disciple
Joined: 05 Jul 2004
Posts: 199
Location: Detroit, MI
|
I think I see. We thought that we could set something on the queue that would automatically check dates and get rid of "old" stuff. I can see that we need to set something on the message to achieve this. I'll check to see if that's been done properly.
So far as "getting lost", our system sends messages to other systems, in particular a FAXing subsystem, an e-mail infrastructure and printers.
Each and all of these can have issues. Printers do occasionally "lose" stuff, and e-mail frequently goes amiss once it's sent out to the Internet (spam filters eat things, etc.). FAXes can go out, but have a print issue.
So, the "amiss" on the items is really related to these services that our system makes use of. |
|
| Back to top |
|
|
| klamerus |
| Posted: Wed May 11, 2005 5:13 pm Post subject: Anything else |
|
|
Disciple
Joined: 05 Jul 2004
Posts: 199
Location: Detroit, MI
|
Well, interestingly enough, the messages are being set with an expiry, which appears to be 1008000. If I divide this by 7 (days/week), 24 (hrs/day), and 60 (min/hr), I end up with 100, which seems like an odd number.
I would expect to still have minutes/hour and then milliseconds. Perhaps someone's math is off (this was also on our non-production site. I'm betting the values are right on production.
I just happened to be watching the queue, as the time for one of these messages to expire passed, and it did disappear off the top of the queue.
So, I'm wondering if there is anything additional that may need to be done to empty the queue of expired items. Should they be automatically removed and nothing additional required? |
|
| Back to top |
|
|
| klamerus |
| Posted: Wed May 11, 2005 5:14 pm Post subject: amendment |
|
|
Disciple
Joined: 05 Jul 2004
Posts: 199
Location: Detroit, MI
|
Sorry,
As pointed out a couple of messages back, I do need to do something.
Do we need to write a small application that will browse down the entire length of the queue (say daily) to "clean off" the messages that have expired?
Thanks, again. |
|
| Back to top |
|
|
| klamerus |
| Posted: Wed May 11, 2005 5:36 pm Post subject: Different expiry |
|
|
Disciple
Joined: 05 Jul 2004
Posts: 199
Location: Detroit, MI
|
Okay, I did manage to get to our production servers and see that the value for expiry there is 6048000, which makes sense for us targeting to have the messages expire in a week.
So, it seems our issue is that we need to create something to regularly do browse gets to clear off the "dead wood".
We had gotten "huge", to where we could not write into the queue further, and didn't know we had to "browse" the messages off the queue.
It would be interesting to understand the philosophy of not actually having them disappear without some overt action. I can think of a couple of reasons, but probably not the ones the were used. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed May 11, 2005 7:29 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Once every X hours, start up a little app that does an MQGET for CorrelID = AnIDThatWillNeverBePresent. The app will search the queue top to bottom in a flash, looking for that message. As it "touches" each message to see if it matches, it will force the expired ones to go away.
You can even just use amqsbcg to browse the whole q to accomplish the same thing if you don't want to write anything. Although that will be a lot of MQGETs the QM has to deal with.
What a pain...why can't expired messages just dissapear on their own?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| klamerus |
| Posted: Thu May 12, 2005 2:44 am Post subject: |
|
|
Disciple
Joined: 05 Jul 2004
Posts: 199
Location: Detroit, MI
|
Yes, this is surprising behavior, but I can understand.
I am just an egg, but I seem to get that the general scheme in MQ is that it is a passive storage system. It responds in turn to requests. So, it's no until a program comes in to query things that the logic "kicks in" to clear out the dead wood.
Anyway, it is certainly painful, but your first suggestion sounds the quickest to do from code.
BTW, any idea on what amqsbcg stands for. The mq is clear, but what's the sbcg part represent? |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu May 12, 2005 3:03 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Actually, 'amqs' is the prefix (an mq sample?). bcg is the part that's harder to figure out.
I assume the b is for "browse".
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| klamerus |
| Posted: Thu May 12, 2005 2:14 pm Post subject: |
|
|
Disciple
Joined: 05 Jul 2004
Posts: 199
Location: Detroit, MI
|
It wouldn't surprise me if the author just used their initials or their girl/boy friend's or their bosses, etc.
Sort of like awk on UNIX. |
|
| Back to top |
|
|
|
|
