| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| Handling full queue |
« View previous topic :: View next topic » |
| Author |
Message
|
| Vitor |
 Posted: Wed Jun 13, 2007 5:18 am Post subject: |
 |
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| fatboyjim wrote: |
| It doesn't matter if we lose some messages, we just don't want it to back up at their end so they turn off the feed. |
You've got some odd business requirements. Never heard "yeah, sure, lose some messages if you need to" before! 
| fatboyjim wrote: |
As I understand it, message queue expiry isn't ideal in this situation as they are not removed from the queue by itself, and you have to perform GETs on the messages which then clear them off the queue |
I believe that expired messages don't count towards depth (I'll stand correction on that). What I do know for sure is that the first GET on the queue (even a browse) triggers the removal of all the expired messages on the queue. This as fast, and more reliable, than a CLEAR QLOCAL seems to be for you.
| fatboyjim wrote: |
I did try (and fail) to setup a dead letter queue, I set the Dead Letter Queue option on the Queue Manager to be the name of a local queue, with attributes derived from SYSTEM.DEAD.LETTER.QUEUE but when a message is put onto a full queue the put just fails, rather than succeeding with the message going onto the DLQ |
Sounds like it's been set up wrong to me. You're probably better off in this instance to increase max depth and keep them all on one queue, than divert the remnents onto a DLQ.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
 |
| Vitor |
| Posted: Wed Jun 13, 2007 5:20 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Vitor wrote: |
Sounds like it's been set up wrong to me. You're probably better off in this instance to increase max depth and keep them all on one queue, than divert the remnents onto a DLQ. |
Unless you're planning to use a triggered dead letter handler to clear them off? Might work, but not functionally different to clearing the original queue down.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fatboyjim |
| Posted: Wed Jun 13, 2007 5:45 am Post subject: |
|
|
Novice
Joined: 25 Jul 2006
Posts: 12
|
| Vitor wrote: |
| fatboyjim wrote: |
| It doesn't matter if we lose some messages, we just don't want it to back up at their end so they turn off the feed. |
You've got some odd business requirements. Never heard "yeah, sure, lose some messages if you need to" before!
|
OK basically what it is is, messages come onto the queue, from a 3rd party feed.. if the application we've got reading from the queue crashes then the queue will fill up, will back out at the 3rd party end, they notice the problem and turn the feed off. The messages are only applicable whilst the application is running, so once the app crashes we can start again from any message (losing thousands doesn't matter as it needs to deal with them in real time)
Hopefully that makes more sense
I got the dead letter queue kind of working now, but it does seem very slow indeed to put messages on, and still backs up at the 3rd party end, so think I will give up on that avenue
You mentioned setting message expiry at this end (we can't really get the 3rd party to change their put options), how do I go about doing this? If you are right in saying expired messages do not count towards queue depth and one get gets rid of all expired messages this would be perfect, we could just up the queue depth and set a timeout of a couple of minutes
Thanks again for your time
Jim |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jun 13, 2007 6:09 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| fatboyjim wrote: |
You mentioned setting message expiry at this end (we can't really get the 3rd party to change their put options), how do I go about doing this?
|
All you need is a scrap of code that reads in the message, sets the expiry and writes the message out. About a dozen lines of code and will move like a train, with no measurable impact on performance especially if the messages are non-persistent (I would imagine they are if they have such limited value).
If it was me, I'd point out to your 3rd party that if they're upset about messages backing up their best course of action is to set an expiry time. It's a one-line change their end with no impact on their processing whatsoever. Everyone's a winner.
| fatboyjim wrote: |
If you are right in saying expired messages do not count towards queue depth and one get gets rid of all expired messages this would be perfect, we could just up the queue depth and set a timeout of a couple of minutes
|
I can't find this documented anyplace so I might have dreamed it. Certainly it's the case once a single GET is performed on the queue the messages are logically deleted.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Jun 13, 2007 7:07 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
The rule with expired messages is that they will never be available for a GET.
Everything else is undocumented - i.e., it works however the developers feel it should work, and don't count on it.
The behavior of the qmgr in regards to expired messages changed between version 5 and version 6, for example. In version 6, there is a background task that will clean up expired messages on a queue. I believe it's undocumented, and I believe it's not tuneable as to how often.
In version 5 the rule used to be that you had to try to GET every expired message in order for messages to be pulled. But that changed during the course of v5, too, I believe.
Otherwise, yes. Have the sender change the expiry. Or write a very simple program (possibly less than 12 lines with MQPUT1) that reads every message off a queue, and puts it to the next queue down the line with an expiry.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Jun 13, 2007 1:16 pm Post subject: |
|
|
Guest
|
"Did you restart the qmgr after setting up the DLQ?....you need to"
Not so. Altering the qmgr object to know the DLQ name is immediate.
An mqput to a local queue that is full will return a reason code/completion code to the application WITHOUT putting a message to the queue. The deat-letter queue is not involved in this situation. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jun 13, 2007 2:59 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| bruce2359 wrote: |
| An mqput to a local queue that is full will return a reason code/completion code to the application WITHOUT putting a message to the queue. The deat-letter queue is not involved in this situation. |
I got the impression this lot were feeding through a channel?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fatboyjim |
| Posted: Thu Jun 14, 2007 12:45 am Post subject: |
|
|
Novice
Joined: 25 Jul 2006
Posts: 12
|
Yes the messages are coming through a channel, it is working fine now (after a restart)
What I have finally implemented is:
Message expiry at the 3rd party end so that a full queue can be cleared easily the next time our application starts up after closing down and letting the queue fill up for whatever reason.
A DLQ for messages that cannot go onto the main queue to be put onto.
A batch file which runs the runmqdlq handler, with rules as such:
- If reason: MRQC_Q_FULL -> discard message
- else: FWD message to investigation queue.
I have set the batch file to run as a Queue Manager service, and everything's working fine!
Thanks for all of the tips. Everyone has been of great help
Jim |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Jun 14, 2007 3:34 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
You probably want to change the batch file to be a Scheduled Task instead of a Queue Manager Service.
You don't want to leave a DLQ handler running constantly. What if something it does causes a message to go on the DLQ...?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
