| Author |
Message
|
| kash3338 |
 Posted: Wed Oct 08, 2014 4:01 am Post subject: Retry subcomponent - WMB |
 |
|
Shaman
Joined: 08 Feb 2009
Posts: 709
Location: Chennai, India
|
Hi,
I am aware this topic has been discussed extensively over the years.
I was not able to conclude if my design is correct or incorrect. Hence posting this query.
If the same has been clarified in this forum earlier, my apologies for missing out on checking that. I would be glad to know here.
I have designed a retry sub-component as below. Just want to know if this is a good design or if there are any better solution for this.
Requirement -
When there is a failure in connecting to end application, I would need to perform a retry n number times with x seconds of time interval between each try. Both n and x are configurable at flow level.
Solution -
1. I have two UDP variables - No. of retries (n) and Time Interval (x) between retries.
2. When failure occurs, I set the following,
MQMD.Expiry to x
MQRFH2.usr.RetryCount to n (logic to check for retry count is taken care)
ReplyToQueue to source queue (Input Queue) of my flow
MQMD.Report to "MQRO_EXPIRATION_WITH_FULL_DATA"
Now, after the expiry, the report message (the actual payload) will be pushed back to the source queue and the retry happens. Is this approach Ok/Good/Bad or any better approach? |
|
| Back to top |
|
 |
| McueMart |
| Posted: Wed Oct 08, 2014 4:11 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
You are right, it has been discussed quite a lot 
I personally like to use a DB component which stores the payload along with the 'last retry time' and 'rety count'. Then you have a 'retry thread' triggered by possibly an Automatic Timer node (running every few seconds).
Saying that, I imagine your implementation would work. Are you putting the 'messages to retry' to some temporary queue while they wait to expire? I'm not exactly sure how it works, but doesnt MQ require an attempt to MQGET off the queue for the messages to actually expire (and cause the expiration message to be sent) ? |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Oct 08, 2014 4:50 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| McueMart wrote: |
| I'm not exactly sure how it works, but doesnt MQ require an attempt to MQGET off the queue for the messages to actually expire (and cause the expiration message to be sent) ? |
Not in WMQv7.0 or later, but this IMHO is a weakness. The scavenger process (which expires messages an MQGET has not) runs at a hard wired interval (IIRC 30 minutes) and you won't get the expiry report until it runs. This could yield a longer delay than desired.
Before someone corrects me, I'm assuming the OP is using distributed and conceed that the scavenger process on z/OS runs at a user configurable interval.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| kash3338 |
| Posted: Wed Oct 08, 2014 5:30 am Post subject: |
|
|
Shaman
Joined: 08 Feb 2009
Posts: 709
Location: Chennai, India
|
| McueMart wrote: |
| I personally like to use a DB component which stores the payload along with the 'last retry time' and 'rety count'. Then you have a 'retry thread' triggered by possibly an Automatic Timer node (running every few seconds). |
I had an idea about this method, but felt that timer node polling every second is not the right approach to be designed. The flow will be deployed along with the other main flows and no point of having the polling per second option.
| McueMart wrote: |
| Saying that, I imagine your implementation would work. Are you putting the 'messages to retry' to some temporary queue while they wait to expire? I'm not exactly sure how it works, but doesnt MQ require an attempt to MQGET off the queue for the messages to actually expire (and cause the expiration message to be sent) ? |
Yes. I am putting that to a temp Queue. I thought MQ v7.0 or later had this option configurable at QM level (setting the ExpiryInterval option). But not clear on that though.
| Vitor wrote: |
| Before someone corrects me, I'm assuming the OP is using distributed and conceed that the scavenger process on z/OS runs at a user configurable interval. |
It is not a distributed environment. It is in Windows OS. It is in IIB v9. Is that not possible in this environment? I was not clear even after looking at many posts around this. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Oct 08, 2014 6:21 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| kash3338 wrote: |
| Vitor wrote: |
| Before someone corrects me, I'm assuming the OP is using distributed and conceed that the scavenger process on z/OS runs at a user configurable interval. |
It is not a distributed environment. It is in Windows OS. It is in IIB v9. Is that not possible in this environment? I was not clear even after looking at many posts around this. |
In my world, there is z/OS and there is distributed. So to be clear only z/OS allows the scavanger process to be configured; on all other platforms it's hard wired.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| kash3338 |
| Posted: Wed Oct 08, 2014 6:31 am Post subject: |
|
|
Shaman
Joined: 08 Feb 2009
Posts: 709
Location: Chennai, India
|
| Vitor wrote: |
| So to be clear only z/OS allows the scavenger process to be configured; on all other platforms it's hard wired. |
Thanks Vitor. But referring this link here it says,
| Quote: |
| On distributed platforms, 'ExpiryInterval' can be added to the 'TuningParameters' stanza in the queue manager's qm.ini file |
|
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Oct 08, 2014 7:07 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| kash3338 wrote: |
Thanks Vitor. But referring this link here it says,
| Quote: |
| On distributed platforms, 'ExpiryInterval' can be added to the 'TuningParameters' stanza in the queue manager's qm.ini file |
|
Well now that's something I've learned today!
So you can increase the scavenger's frequency. But be aware that's a queue manager wide setting; if you want the expiry message to be produced close to x seconds, you'll need it to be running every 2 seconds or so, and it will run every 2 seconds on every local queue hosted by that queue manager. So if there are a lot of queues, the queue manager is going to burn a lot of cycles looking for expired messages at the expense of actually processing messages.
Something to be aware of.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| kash3338 |
| Posted: Wed Oct 08, 2014 7:57 pm Post subject: |
|
|
Shaman
Joined: 08 Feb 2009
Posts: 709
Location: Chennai, India
|
| Vitor wrote: |
So you can increase the scavenger's frequency. But be aware that's a queue manager wide setting; if you want the expiry message to be produced close to x seconds, you'll need it to be running every 2 seconds or so, and it will run every 2 seconds on every local queue hosted by that queue manager. So if there are a lot of queues, the queue manager is going to burn a lot of cycles looking for expired messages at the expense of actually processing messages.
Something to be aware of. |
Totally agree on this point. It is because of this point that I had a concern on this design. Any alternate for this approach or any other better approach for Retry? Just to clarify, we do not have Database used in any of our services and hence I do not want to introduce Database just for this component. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Oct 08, 2014 8:05 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20697
Location: LI,NY
|
Avoid burning a lot of cycles...
Use following set up: Put a long running program that does a get with wait of a message with message ID or correlation ID on the queue that is never likely to happen...
Like start with a put retrieve that message by message id and then do a get with wait in a Loop that looks for that same message. The get with wait is never likely to return a message (no match on messageid after the first time) but I expect it will ensure a timely expiration on your messages...
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| kash3338 |
| Posted: Wed Oct 08, 2014 9:33 pm Post subject: |
|
|
Shaman
Joined: 08 Feb 2009
Posts: 709
Location: Chennai, India
|
| fjb_saper wrote: |
| Avoid burning a lot of cycles... |
| fjb_saper wrote: |
Use following set up: Put a long running program that does a get with wait of a message with message ID or correlation ID on the queue that is never likely to happen...
Like start with a put retrieve that message by message id and then do a get with wait in a Loop that looks for that same message. The get with wait is never likely to return a message (no match on messageid after the first time) but I expect it will ensure a timely expiration on your messages... |
I am sorry, I am confused with what is suggested. Can you please explain in a different way?  |
|
| Back to top |
|
|
| kash3338 |
| Posted: Wed Oct 08, 2014 10:37 pm Post subject: |
|
|
Shaman
Joined: 08 Feb 2009
Posts: 709
Location: Chennai, India
|
| kash3338 wrote: |
| fjb_saper wrote: |
Use following set up: Put a long running program that does a get with wait of a message with message ID or correlation ID on the queue that is never likely to happen...
Like start with a put retrieve that message by message id and then do a get with wait in a Loop that looks for that same message. The get with wait is never likely to return a message (no match on messageid after the first time) but I expect it will ensure a timely expiration on your messages... |
I am sorry, I am confused with what is suggested. Can you please explain in a different way? |
When I read it for many more times, I was able to debug your statements 
Please confirm if my understanding is correct?
The flow design for retry is like this,
| Code: |
| MQINPut --> Compute --> MQGet --> MQOutput |
Drop my message to be replayed to a flow in Input Queue.
Get the MessageId of that message
Enable Get by MessageId option in MQGet and set the wait interval to the time interval required
The MQGet will wait for such long and then process the message to replay. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Oct 09, 2014 4:15 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20697
Location: LI,NY
|
Sorry but no, my reflection had absolutely nothing to do with IIB or WMB. It was solely geared towards a cheap way of making sure your messages expire as soon as possible...
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| kash3338 |
| Posted: Thu Oct 09, 2014 8:57 pm Post subject: |
|
|
Shaman
Joined: 08 Feb 2009
Posts: 709
Location: Chennai, India
|
| fjb_saper wrote: |
| Sorry but no, my reflection had absolutely nothing to do with IIB or WMB. It was solely geared towards a cheap way of making sure your messages expire as soon as possible... |
OK. But if the Queue is made to be hooked with a GET over a loop or some other way, I don't think that's the best of solution.
Is there no other way or no other better approach for retry? I could think few with use of Timer nodes or using a DB, but I don't want to implement that way as explained before. Any other efficient approach? |
|
| Back to top |
|
|
| kash3338 |
| Posted: Fri Oct 10, 2014 3:36 am Post subject: |
|
|
Shaman
Joined: 08 Feb 2009
Posts: 709
Location: Chennai, India
|
I tried exploring few other options as well and I am not able to think of one simple approach to perform a retry in WMB (with configurable "No. of retries" and "Time interval between retries" properties). I will be able to achieve them using a Java Singleton or using Database, but with the use of native WMB features and WMQ, there is no simple design (AFAIK).
I dont want to use Java Singleton and Database approach to keep my retry component pretty simple. Timer nodes have a performance bottleneck. Any experts here who can suggest a better approach to retry using native features of WMB/WMQ? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Oct 10, 2014 5:55 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20697
Location: LI,NY
|
The message time out suggestion was not so bad. Only to make it effective you want a thread, running outside of the broker doing a get with wait in a loop. The wait interval will determine the expiry precision. See my comments above...
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
