| Author |
Message
|
| geewee |
 Posted: Mon Oct 01, 2012 6:10 am Post subject: controlling redelivery delay on message/queue or mqget/mqput |
 |
|
Apprentice
Joined: 31 Jul 2012
Posts: 28
|
I will give a quick requirement overview:
We have a MB flow that post messages to a web service.
If the web service is unavailable we move the messages to an error queue, for later processing.
We are now implementing a flow to pick messages from this error queue, and do redelivery.
We want to control redelivery delay, and redelivery count.
The number of messages on error queue can be quite large, (~1 000 000) so we would like to limit the cpu usage and appliation logic involved in
iterating messages before doing redelivery. Leaving the messages idle on queue between retries is assumed to be more lightweight than scanning through messages to look for some mq header properties.
Are there any possibilities to control the redelivery delay, either per message, per queue, or on the mq get/mq put?
redelivery delay and redelivery limit are supported by some of the MDB containers |
|
| Back to top |
|
 |
| lancelotlinc |
| Posted: Mon Oct 01, 2012 6:30 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
Here is something you may like to try. Set the CorrelId to a value based on time range. Use GetByCorrelId option to retrieve the messages that have been idle for the equivalent time. Once an hour/day, get all messages to clean up.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Oct 01, 2012 6:48 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
you could construct the message to have an expiry value and report options to provide the full message in the expiration report, and then cause the expiration report to be delivered to the input queue again, and then write the messages to a queue that never reads them (so they sit and expire).
But in general, this is a troublesome pattern you're trying to implement. If you're delaying lots of messages over a long period of time, it's much better to halt the process that creates the requests in the first place rather than waiting for the service to recover. |
|
| Back to top |
|
|
| geewee |
| Posted: Mon Oct 01, 2012 10:12 am Post subject: |
|
|
Apprentice
Joined: 31 Jul 2012
Posts: 28
|
I will look into both the correlation id and the expiry report possibilities.
We do not want to stop the incoming messages even if we are not able to post them to the web service. We will then take effort of queuing them, for later posting.
This is of course a error handling pattern, and high availability of web service is also part of solution. But we cannot stop customers from posting requests even if we are not able to pass them on to the back end system at the moment. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Oct 01, 2012 10:47 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
Make sure your disk space is ample and your max queue depths are sprite.
An expansion of the CorrelId concept is to store in a db table all the MsgIds needing to be reprocessed. Then remove each Id row-by-row as that MsgId is successfully processed.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| NealM |
| Posted: Mon Oct 01, 2012 12:19 pm Post subject: |
|
|
Master
Joined: 22 Feb 2011
Posts: 230
Location: NC or Utah (depends)
|
mqjeff's suggestion to use an expiry report is neat but at the same time a bit tricky. Once a message on a queue expires, nothing happens report-wise until something else touches the queue, then all the expired messages are sent to their reply-to queues (if possible). This may be fine if there is a steady stream of activity to the queue, otherwise an expired message may sit there for a long time before a report is generated. So you might set your expiries for say 30 minutes and depending on your business, a whole bunch of last Friday's expires get "reported" on Monday morning.
As a 3rd choice (from mqjeff and lancelot's both workable suggestions), Since you want to control delay and # times to re-apply, you might want to look into a re-apply flow that is Timeout node based and using either an RFH2 header or an extra message body element (that would need to be stripped) with a counter to be incremented/decremented. Or with a little care, misappropriate the MQMD.Feedback field as your counter. |
|
| Back to top |
|
|
| geewee |
| Posted: Mon Oct 01, 2012 11:47 pm Post subject: |
|
|
Apprentice
Joined: 31 Jul 2012
Posts: 28
|
I must admit I am a bit surprised that this pattern is not supported from Message Oriented Middleware.
MOM gives loose coupling between sending system and receiving system. Let us say the transaction fails on receiving side, and message is backed out to queue. Will it just get picked up straight away for retry of transaction without any kind of delay?
NealM, I am considering this alternative, but if the redelivery delay is within RFH2 I believe the consuming application (ReApplyFlow) will have to browse messages on queue, looking for the next messages to get. Frequency of this message browsing can be configured to limit cpu consumption, but still it will have a cost.
As expiry of messages is part of mq behaviour I was assuming delivery delay of messages was also possible. |
|
| Back to top |
|
|
| Esa |
| Posted: Tue Oct 02, 2012 12:28 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
Have you considered using Timeout nodes?
I implemented someting very similar for a customer last spring and got it working nicely with Timeout nodes. The resulting flow was a bit complex, I must admit. And the Timeout nodes are a bit tricky.
The main idea was that application would stop trying the web service if it noticed that it was down and start putting all new messages directly in the retry queue. The delayed retry flow is triggered via a TimeoutControl/TimeoutNotification pair. While the service is down, it will just keep re-triggering itself. When the service is up again, the retry flow will process all the messages on the retry queue and then set a shared varible that will tell the main flow that it can start calling the web service directly again. For this to work both flows will have to be put in the same flow file.
Something like that. Actually my application was even more complex, because it was implementing a pub/sub pattern for web services as well. |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Oct 02, 2012 1:33 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
If you want to pace the re-delivery, you simply browse the queue and inspect the first message's put date/time.
If the date/time (compared to the current date/time) is too soon for redelivery simply sleep for a period, and then reset the browse cursor and browse again.
Repeat until you decide to process the message and then destructively get the message under the browse cursor.
This will also work in a multi-thread environment if you lock the message under the browse cursor. |
|
| Back to top |
|
|
|
|
