| Author |
Message
|
| fatboyjim |
 Posted: Tue Jun 12, 2007 2:01 am Post subject: Handling full queue |
 |
|
Novice
Joined: 25 Jul 2006
Posts: 12
|
All,
I have a C# application that reads messages from a WebSphere MQ queue with a depth of 5000, messages are being pumped onto this queue by a third party
Unfortunately, it is not possible for my app to be running all of the time, and with roughly 10 messages a second coming in it fills up rather quickly, then the third party system backs off and we have to contact them to switch their feed back on
Is there something I can do to prevent this? Something along the lines of:
- Allow WebSphere to continue to receive messages once the queue is full, but just ditch them
or
- When the queue is nearly full, remove the oldest 1000 messages from the queue
Whatever I do, it needs to be tied in with WebSphere in some way - so that it is always running whilst the queue can accept messages (if it is simply another application, it is likely it won't be running all of the time either)
Thanks very much for any help
Jim |
|
| Back to top |
|
 |
| elvis_gn |
| Posted: Tue Jun 12, 2007 2:06 am Post subject: |
|
|
Padawan
Joined: 08 Oct 2004
Posts: 1905
Location: Dubai
|
Hi fatboyjim,
I guess, simplest would be to trigger the queue on 'x' number of messages and run an app to delete the first 1000 msgs.
Regards. |
|
| Back to top |
|
|
| fatboyjim |
| Posted: Tue Jun 12, 2007 2:12 am Post subject: |
|
|
Novice
Joined: 25 Jul 2006
Posts: 12
|
Elvis lives! Thanks for the message pal, is there some documentation somewhere to explain how to run apps from triggers? I've been unsuccessful in my searching so far (the IBM page I found on google seems to have been deleted)
Thanks
Jim |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jun 12, 2007 2:15 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| fatboyjim wrote: |
| Elvis lives! Thanks for the message pal, is there some documentation somewhere to explain how to run apps from triggers? |
The Application Programming Guide has a whole chapter devopted to triggering. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fatboyjim |
| Posted: Tue Jun 12, 2007 2:18 am Post subject: |
|
|
Novice
Joined: 25 Jul 2006
Posts: 12
|
Thanks for that, I was looking on the website
I do have all of the PDFs downloaded, but hadn't looked through each and every one (found the Application Programming Guide now - named - obviously, csqzal09.pdf)
I have also found the chapter you mention
Thanks for your time
Jim |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jun 12, 2007 2:24 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
|
| Back to top |
|
|
| fatboyjim |
| Posted: Tue Jun 12, 2007 3:11 am Post subject: |
|
|
Novice
Joined: 25 Jul 2006
Posts: 12
|
Lovely, thanks for taking the time to find that for me
I am still ever so slightly lost, there are some points perhaps you could help clear up for me:
Ultimately, do you think, given my original post, that a trigger is the best way forward? Is there no better way to either accept and ditch incoming messages?
If it is, am I right in understanding I need to have a trigger monitor application running the whole time, and another queue where the trigger will put messages on to? If this is the case (and the application needs to be written by myself) this isn't really acceptable as it may not always be running. I really need something that's integrated with MQ - apologies if I have misunderstood
I also see the possibility of having the trigger fire an application off, am I therefore able to, instead of having the initiation queue and trigger monitor application running, in MQ Explorer set:
Trigger Control: On
Trigger Type: Depth
Tripper Depth: 4500
Trigger Message Priority: 0
Trigger Data: <blank>
Initiation queue: <blank>
Process name: PROCESS.EXE
And have PROCESS.EXE started whenever the queue depth reaches 4500? This application could then connect to the queue and delete the oldest 1000 messages from the queue?
If this is the case, where must PROCESS.EXE reside? I cannot find a sensible place to put it in the Program Files folder, and the documentation doesn't seem to cover this. I did think about supplying the full path to the file, but the field will not allow colons or backslashes
Thanks again for your time, it is appreciated
Jim |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Jun 12, 2007 3:18 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
read the manuals. Process name does not refer to your application but to a definition of your application in MQ. Read up on Define process.... 
And remember the difference between the trigger monitor and the client trigger monitor...
Anyways with the volume you are getting I would suggest running the app as a client, on 2 different servers. This way you can take one down for maintenance while the other is still processing the messages...
Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jun 12, 2007 3:21 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| fatboyjim wrote: |
| Ultimately, do you think, given my original post, that a trigger is the best way forward? Is there no better way to either accept and ditch incoming messages? |
I certainly do! 
| fatboyjim wrote: |
If it is, am I right in understanding I need to have a trigger monitor application running the whole time, and another queue where the trigger will put messages on to? If this is the case (and the application needs to be written by myself) this isn't really acceptable as it may not always be running. I really need something that's integrated with MQ - apologies if I have misunderstood |
A trigger monitor is supplied as part of MQ - runmqtrm IIRC but look it up. Most people use that unmodified and leave it running for as long as the queue managers up. What do you see as unacceptable here?
| fatboyjim wrote: |
If this is the case, where must PROCESS.EXE reside? I cannot find a sensible place to put it in the Program Files folder, and the documentation doesn't seem to cover this. I did think about supplying the full path to the file, but the field will not allow colons or backslashes
|
It can reside anywhere accessable to the trigger monitor (because the trigger monitor needs to find it to start it). The field not allowing colons or backslahes is news to me; my process definitions have them!
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Jun 12, 2007 6:14 am Post subject: |
|
|
Guest
|
| You could increase maxqdepth to 10,000 or 20,000 - something that will accomodate all the inbound messages. |
|
| Back to top |
|
|
| PisgahMan |
| Posted: Tue Jun 12, 2007 9:51 am Post subject: |
|
|
Voyager
Joined: 27 Jul 2004
Posts: 93
|
| Thats what I was thinking, if you concern is filling up the queue can you increase the maxdepth? |
|
| Back to top |
|
|
| fatboyjim |
| Posted: Wed Jun 13, 2007 4:54 am Post subject: |
|
|
Novice
Joined: 25 Jul 2006
Posts: 12
|
| Vitor wrote: |
| fatboyjim wrote: |
| Ultimately, do you think, given my original post, that a trigger is the best way forward? Is there no better way to either accept and ditch incoming messages? |
I certainly do!
| fatboyjim wrote: |
If it is, am I right in understanding I need to have a trigger monitor application running the whole time, and another queue where the trigger will put messages on to? If this is the case (and the application needs to be written by myself) this isn't really acceptable as it may not always be running. I really need something that's integrated with MQ - apologies if I have misunderstood |
A trigger monitor is supplied as part of MQ - runmqtrm IIRC but look it up. Most people use that unmodified and leave it running for as long as the queue managers up. What do you see as unacceptable here?
| fatboyjim wrote: |
If this is the case, where must PROCESS.EXE reside? I cannot find a sensible place to put it in the Program Files folder, and the documentation doesn't seem to cover this. I did think about supplying the full path to the file, but the field will not allow colons or backslashes
|
It can reside anywhere accessable to the trigger monitor (because the trigger monitor needs to find it to start it). The field not allowing colons or backslahes is news to me; my process definitions have them! |
Thanks for the message. I completely understand what's going on now, I was just slightly overwhelmed by it all. Thanks again for all of your help
I now have my queue set to trigger on a certain depth, which runs a batch file. The batch file clears the queue and turns triggering for the queue back on. This works when using test messages, but doesn't work when there is a steady flow of messages coming onto the queue. Is this a limitation of CLEAR.QLOCAL('QueueName') ? (it seems to try to clear - a few messages disappear - then give up as soon as another message comes onto the queue). Is there a better way to do this?
And to those who suggested just increasing the max queue depth, thanks for the suggestion, unfortunately it's no use here as the messages just keep on coming
Thanks for all of your help
Jim |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jun 13, 2007 4:58 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| fatboyjim wrote: |
| Is there a better way to do this? |
Taking a step back, what is the point to these messages if they can be discarded when the queue's too full? If it's that certain that these messages are unecessary, why are they sent? If they're no longer necessary due to the passing of time, have you considered setting the expiry (ideally at the 3rd party end but possibly yourself)?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fatboyjim |
| Posted: Wed Jun 13, 2007 5:07 am Post subject: |
|
|
Novice
Joined: 25 Jul 2006
Posts: 12
|
| Vitor wrote: |
| fatboyjim wrote: |
| Is there a better way to do this? |
Taking a step back, what is the point to these messages if they can be discarded when the queue's too full? If it's that certain that these messages are unecessary, why are they sent? If they're no longer necessary due to the passing of time, have you considered setting the expiry (ideally at the 3rd party end but possibly yourself)? |
It's not that the messages are unnecessary, it's just that the 3rd party gets annoyed when the messages don't come through to our queue manager. It doesn't matter if we lose some messages, we just don't want it to back up at their end so they turn off the feed.
As I understand it, message queue expiry isn't ideal in this situation as they are not removed from the queue by itself, and you have to perform GETs on the messages which then clear them off the queue
I did try (and fail) to setup a dead letter queue, I set the Dead Letter Queue option on the Queue Manager to be the name of a local queue, with attributes derived from SYSTEM.DEAD.LETTER.QUEUE but when a message is put onto a full queue the put just fails, rather than succeeding with the message going onto the DLQ |
|
| Back to top |
|
|
| kevinf2349 |
| Posted: Wed Jun 13, 2007 5:10 am Post subject: |
|
|
Grand Master
Joined: 28 Feb 2003
Posts: 1311
Location: USA
|
| Quote: |
| I did try (and fail) to setup a dead letter queue, I set the Dead Letter Queue option on the Queue Manager to be the name of a local queue, with attributes derived from SYSTEM.DEAD.LETTER.QUEUE but when a message is put onto a full queue the put just fails, rather than succeeding with the message going onto the DLQ |
Did you restart the qmgr after setting up the DLQ?....you need to |
|
| Back to top |
|
|
|
|
