| Author |
Message
|
| m.schneider |
 Posted: Fri Nov 16, 2007 3:18 am Post subject: Design a retry logic with time scale |
 |
|
Centurion
Joined: 10 Apr 2007
Posts: 132
Location: Germany
|
I'm trying to implement a flow that requeues a message. There should also be a rising wait time for the retry depending on a retry counter.
The business case is that I receive messages which I process over a database to a backend system. Now I have to look up in the database the answer from the backend system. The answer is unique for every processed message. In best case the reply is within a second there but it could also be time x.
My Ideal would be to look up periodicaly, countng up a retry-counter. Depending on this counter I would feed three queues for example.
First queue is attached to a flow that looks up every 10 seconds, second queue every 30s etc., ...
To implement these flow I would use an automatic timernode which uses a get node to get the messages of the queue and connect it to an outputnode. I would also attach the outputnode-out to the getnode-in so that in a loop every message gets processed.
Problem is that these flow would also start every x seconds if there are now messages in the queues.
By using controlled timernodes it doesn't get better, because after you processed the first message and send a timeoutRequest to the timernode to start in time x, the next message arrives and there would be another timeoutRequest, ...
The most examples are cut out just for processing one message at a time llike here:
http://www.ibm.com/developerworks/websphere/library/techarticles/0603_schutz/0603_schutz.html
http://www.ibm.com/developerworks/websphere/library/techarticles/0601_cox/0601_cox.html
Can someone help out? |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Fri Nov 16, 2007 3:51 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
You might want to look at it as an aggregation.
Have the database create the reply message and put it to MQ.
Now you have the aggregation...Use max timeout on aggregation.
Whenever the message comes in it will be aggregated...
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| m.schneider |
| Posted: Fri Nov 16, 2007 4:07 am Post subject: |
|
|
Centurion
Joined: 10 Apr 2007
Posts: 132
Location: Germany
|
| Thanks for the quick reply. But I can't change anything on the database, ... |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Nov 16, 2007 4:12 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I'm confused as to why a straight timoutcontrol/timeoutnotification can't work.
You say
| Quote: |
| By using controlled timernodes it doesn't get better, because after you processed the first message and send a timeoutRequest to the timernode to start in time x, the next message arrives and there would be another timeoutRequest, ... |
You can do the following:
1) send the messsge to the mainframe
2) check for the reply, if not there
3) send a timeout message which includes enough information from the input message to identify the reply with a 10 second timeout
4) the TimeoutNotificaiton node will fire in 10 seconds
5) before then, another message arrives at 1).
6) The timeout notification node fires for the first message
7) You check for the right reply, based on the data in the TimeoutNotification message.
 If it's not there, decide how long to wait again (10 seconds, 30 seconds, etc.), and send another timeout request with the original request information
9) The timeout notification fires for the second message, go to 7
10) The timeout notification fires for the first message, AGAIN. Go to 7
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Nov 16, 2007 4:25 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| m.schneider wrote: |
| Thanks for the quick reply. But I can't change anything on the database, ... |
Don't change anything on the database... just front end it with a request / reply type of service... So you receive a request and process it. Send the reply ..., as long as you have enough information sending the reply could be asynchronous...
What type of process /state does the dB perform?
If it is completely async try this
Send an aggreg request. Store the reply info to the DB correlating it to the request and process info ...(add table) and process the message. When the reply gets there correlate it to the request and send the reply fan in message. 
Or run a batch process every so often to check and correlate non sent answers and send the reply message if reply info is there...  
In other words make it a push process instead of a pull process that you need to constantly poll...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| m.schneider |
| Posted: Fri Nov 16, 2007 5:21 am Post subject: |
|
|
Centurion
Joined: 10 Apr 2007
Posts: 132
Location: Germany
|
@jefflowrey thanks for your reply
problem with controlled-timernodes is that you can't process messages with the same uid parallel. While the first message is waiting, the second arrives with the same uid.
Now what happens:
a) you configured the request with allowoverwrite=no -> second msg runs into an error
b) you configured the request with allowoverwrite=yes -> to the waiting time are ten more seconds added, ... bad if a lot of msg's arrive
This is my understanding, perhaps I'm getting this wrong, ... I'm glad if you have a solution that works.
My idea was to use multiple timernode as a trigger for flows that use a getnode to processes all msg's in the delay-xsec-queue. |
|
| Back to top |
|
|
| m.schneider |
| Posted: Fri Nov 16, 2007 5:29 am Post subject: |
|
|
Centurion
Joined: 10 Apr 2007
Posts: 132
Location: Germany
|
@fjb_saper
My problem is the polling, ... But you get it right, ... I want to provide a asynchronos mq-based facade for the database.
| Quote: |
| Or run a batch process every so often to check and correlate non sent answers and send the reply message if reply info is there... |
This could be a workaround, but it wouldn't speed things up, ... and the batch job would also run if there is nothing to process. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Nov 16, 2007 5:44 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| m.schneider wrote: |
@jefflowrey thanks for your reply
problem with controlled-timernodes is that you can't process messages with the same uid parallel. While the first message is waiting, the second arrives with the same uid. |
No, I don't think so.
I think it's based on the TimeoutRequest message Identifier field, not the UID that associates the Control node with the Notification node.
This could be set to, for example, the DB row id that you need to poll for.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Nov 16, 2007 6:01 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| m.schneider wrote: |
@fjb_saper
My problem is the polling, ... But you get it right, ... I want to provide a asynchronos mq-based facade for the database.
| Quote: |
| Or run a batch process every so often to check and correlate non sent answers and send the reply message if reply info is there... |
This could be a workaround, but it wouldn't speed things up, ... and the batch job would also run if there is nothing to process. |
OK so you have 2 different scenarii:
- Publication of changes to the DB.
This should be messages that get published in the broker. Anybody interested in the change can then subscribe to the topic.
Typically DB trigger or batch process at intervals.
WII comes to mind
- Request / reply scenario.
The hard part here is waiting for the reply. As you work fully asynchronously a good pattern for this is aggregation with 2 flows:
a) fan out flow (can output the original request and the request message)
Time out should be viewed as no longer interested in the reply any more.
b) separate fan in flow
What is important in the request / reply flow when you make it be completely asynchronous is that you find a way to store the request information (msgid , correlid, request info) so that you can later correlate the reply. When the reply comes you can then send the reply message to the fan in queue.
This can be done either by a timed batch process that scans just the new arrivals (i.e. index work to be done on replies) or a DB trigger induced process.
WII can / could help
The reason you want fan in flow and fan out flow to be 2 separate flows is for volume and speed. Having multiple instances and the flows separated allows you to process way more way faster and mostly without any system/design induced delays.
Hope this helps some
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Nov 16, 2007 6:16 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
FJ - You can't retry an aggregation, without using one of the other techniques mentioned.
This also just moves the original problem to the other side of the aggregation flow. The thing sending the reply still has to poll the database at scaling intervals, until the reply data is ready. Or at least poll the database, and send any replies that are ready.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Nov 16, 2007 6:31 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Jeff, it's not about retrying the aggregation. It's about leaving it open until you don't want the answer any more. (Greater timeout than 10 seconds) and not holding back any other message in the input queue while waiting...
So there is never a retry of the aggregation. The pull model has been transformed into a push model....
Sure you still poll the DB. But with the correct indexes this can happen lightning fast.
You can also take advantage of a product like WII which has you completely in push mode... No polling at all, all event driven right ?
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| m.schneider |
| Posted: Fri Nov 16, 2007 8:01 am Post subject: |
|
|
Centurion
Joined: 10 Apr 2007
Posts: 132
Location: Germany
|
| Quote: |
I think it's based on the TimeoutRequest message Identifier field, not the UID that associates the Control node with the Notification node.
This could be set to, for example, the DB row id that you need to poll for. |
Right, I just tried it out and used my row-id as request identifier, ... works good. I got the TimeoutRequest Identifier and the UID mixed up.
But here's the next question during wait time all this messages reside on SYSTEM.BROKER.TIMEOUT.QUEUE
I just used 3 parallel msg's, what about 100, 1000 or more msg's? Does this scale?
Thanks a lot jefflowrey, I really appreciate the solution and learned something  |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Nov 16, 2007 9:04 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| m.schneider wrote: |
| I just used 3 parallel msg's, what about 100, 1000 or more msg's? Does this scale? |
It should scale just as well as Broker does... in the same ways...
I wouldn't try to put 10,000 timeouts to a single instance of a single flow, though. I'd use at least multiple instances there.
If you have deep performance requirements, you can ask your IBM Sales Rep for the right mechanism to get in depth analysis done or questions answered.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
