| Author |
Message
|
| mb01mqbrkrs |
 Posted: Mon Mar 19, 2012 3:28 am Post subject: MQGet node not performing backout? |
 |
|
Apprentice
Joined: 18 Nov 2011
Posts: 48
|
Hi Experts.
I've been performing some testing with the MQGet node, but seem to have a problem with the backout handling...
HTTP Input (just to trigger the flow) -> MQGet-> Throw
Each time the flow is run, the message on the queue that MQGet is configured to use, has its BOCount increased by one - which is what I would expect.
However, the BOCount seems to be able to increase over the BOThreshold for the queue and the message doesn't hit the BOQ.
Any ideas? - or is this by design?
MB 6.1.0.7 |
|
| Back to top |
|
 |
| Esa |
| Posted: Mon Mar 19, 2012 3:43 am Post subject: Re: MQGet node not performing backout? |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
How much over the treshold?
I am not sure, but I think MQInput puts the message in BOQ after getting the message, if it detects that backout count exeeds the treshold. MQGet should do the same. So the message may not end up in the backout queue when the transaction is backed out, because no MQGet node code is not executed when the backout happens. It should happen the next time you try to MQget.
But it's possible that this functionality has not been implemented in MQget node. |
|
| Back to top |
|
|
| mb01mqbrkrs |
| Posted: Mon Mar 19, 2012 3:47 am Post subject: |
|
|
Apprentice
Joined: 18 Nov 2011
Posts: 48
|
| Thanks for the response Esa. To answer your question, the backout count in my testing has increased to more than double that of the backout threshold. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Mar 19, 2012 4:59 am Post subject: Re: MQGet node not performing backout? |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mb01mqbrkrs wrote: |
Any ideas? - or is this by design?
|
What's the transactionality of your flow?
I'd theorise that the message is not places in the backout queue because it's not the driving message of the flow and needs to be in the same queue if the principle flow restarts. Remember the MQGet node is designed to be used to read a specific message from a queue as part of flow processing; it's not intended to read & empty a queue. So it's reasonable to assume that you'd want that specific message to remain on the original queue.
I'd be interested in IBM's opinion on this via a PMR. Please let us know what they say.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Esa |
| Posted: Mon Mar 19, 2012 5:10 am Post subject: Re: MQGet node not performing backout? |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
Precisely.
Why is the driving message written to a backout queue? To be able to resend it manually after the error situation has been corrected, for example. If the side message would be written to a backout queue too, the flow would fail for the resent message.
It is quite usual that messages in the MQGet queue have an expriration set. That will take care of queue cleanup. |
|
| Back to top |
|
|
| mb01mqbrkrs |
| Posted: Mon Mar 19, 2012 5:32 am Post subject: Re: MQGet node not performing backout? |
|
|
Apprentice
Joined: 18 Nov 2011
Posts: 48
|
Thanks for the replies.
The example flow was just a proof of concept to see if it was possible to trigger reprocessing of messages on an error queue (basically allowing retries with a timer gap). Obviously we wouldn't want the same poison message retried infinitely, which is why I was looking at the backout behaviour of the MQGet node.
| Vitor wrote: |
| Remember the MQGet node is designed to be used to read a specific message from a queue as part of flow processing; it's not intended to read & empty a queue. |
Interesting - I'm sure that your right, but I cant see any reference to this in the product documentation and I cant remember this being said in the training course. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Mar 19, 2012 5:37 am Post subject: Re: MQGet node not performing backout? |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mb01mqbrkrs wrote: |
| Vitor wrote: |
| Remember the MQGet node is designed to be used to read a specific message from a queue as part of flow processing; it's not intended to read & empty a queue. |
Interesting - I'm sure that your right, but I cant see any reference to this in the product documentation and I cant remember this being said in the training course. |
The product documentation is focused on what a feature can do, rather than how to use it. I'd have expected some mention of this in the design section of a training course.
You'll find a wealth of discussion in here on the perils of repeatedly calling an MQGet node (i.e. looping round to empty an error queue). Mostly people running out of memory and/or seeing execution groups fail.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Mar 19, 2012 5:41 am Post subject: Re: MQGet node not performing backout? |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mb01mqbrkrs wrote: |
| The example flow was just a proof of concept to see if it was possible to trigger reprocessing of messages on an error queue (basically allowing retries with a timer gap). |
There are a number of discussions in here on various ways of achiving this.
| mb01mqbrkrs wrote: |
| Obviously we wouldn't want the same poison message retried infinitely |
IMHO you wouldn't want it to be retried automatically more than once. If a transient problem is still a problem when you retry, you either have the retry interval set too short or the problem isn't transient!
You also need to factor in such intangibles as how long the business will tollerate the message whizzing round and round your automatic retry, how this affects the life of your message, etc, etc.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Mar 19, 2012 5:44 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
You also need to bear in mind that this kind of record & replay technology is built in to v8, so if this is a key site requirement you might want to consider going to WMBv8 rather than WMBv7.
Because clearly you're not going to be on WMBv6.1.0.7 much longer. In an ideal world you wouldn't be on it now (you'd be on v6.1.0.10).
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mb01mqbrkrs |
| Posted: Mon Mar 19, 2012 5:59 am Post subject: |
|
|
Apprentice
Joined: 18 Nov 2011
Posts: 48
|
| Vitor wrote: |
IMHO you wouldn't want it to be retried automatically more than once. If a transient problem is still a problem when you retry, you either have the retry interval set too short or the problem isn't transient!
You also need to factor in such intangibles as how long the business will tollerate the message whizzing round and round your automatic retry, how this affects the life of your message, etc, etc. |
Unfortunately in this particular case, the message could fail going to our ERP system due to the record being locked. The record could be unlocked "sometime" in the future (this could be hours). Clearly, if I could just use the built-in Broker retry logic I would - but I needed some way of retrying every hour or so.
I understand that the requirements are far less than ideal and have pushed back without success - apparently this functionality is standard from our ERP Vendor!
I'm guessing that I'm going to have to design some custom retry logic using a persistent store - its a shame that the default broker retry logic cant be customised (to include a retry time). Any thoughts about this would be high appreciated. |
|
| Back to top |
|
|
| Esa |
| Posted: Mon Mar 19, 2012 6:02 am Post subject: Re: MQGet node not performing backout? |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| mb01mqbrkrs wrote: |
Thanks for the replies.
The example flow was just a proof of concept to see if it was possible to trigger reprocessing of messages on an error queue (basically allowing retries with a timer gap). Obviously we wouldn't want the same poison message retried infinitely, which is why I was looking at the backout behaviour of the MQGet node.
|
You can isolate processing of the MQGet message (with a TryCatch node, for example) and after an exception examine backout count programatically (against a constant or UDP). Because your flow is transactional, you must put a copy of the poison MQGet message in another queue if the count is exeeded. Then have another flow read it, remove the message with matching id from the MQGet queue and write it to "poison message" queue.
If I understood correctly what you were trying to do. |
|
| Back to top |
|
|
| mqsiuser |
| Posted: Mon Mar 19, 2012 6:04 am Post subject: |
|
|
Yatiri
Joined: 15 Apr 2008
Posts: 637
Location: Germany
|
So there are these properties. And then your ERP-System will read them somehow: Either MQ or JMS (from a queue)... the properties can be retrieved, e.g. by your ERP-Software-Vendor "JMS-Driver"... and this driver can be configured to do "custom" logic based on the "properties"... properties... only some values (no logic from MQ involved!)... The logic is defined by the reading application (your JMS-Driver from your ERP-System).
_________________
Just use REFERENCEs
Last edited by mqsiuser on Mon Mar 19, 2012 6:18 am; edited 2 times in total |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Mar 19, 2012 6:08 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mb01mqbrkrs wrote: |
| Unfortunately in this particular case, the message could fail going to our ERP system due to the record being locked. The record could be unlocked "sometime" in the future (this could be hours). |
So the user community are happy for records to be sitting happy and unprocessed for hours (which can easily turn into days)? What happens when you finally hit a limit of some kind and the message is deemed "poisoned"? Most users of my experience would react along the lines of "the ERP has been locked up for 12 hours? Nothing's going through? Where's the trouble ticket? What do you mean there's no trouble ticket until the automatic retry expires???"
| mb01mqbrkrs wrote: |
| I understand that the requirements are far less than ideal and have pushed back without success - apparently this functionality is standard from our ERP Vendor! |
You need a better ERP vendor. 
| mb01mqbrkrs wrote: |
| I'm guessing that I'm going to have to design some custom retry logic using a persistent store |
 See my comments above; you may find some of the discussion helpful.
| mb01mqbrkrs wrote: |
| its a shame that the default broker retry logic cant be customised (to include a retry time). |
See my comments above re WMBv8.
_________________
Honesty is the best policy.
Insanity is the best defence.
Last edited by Vitor on Mon Mar 19, 2012 6:18 am; edited 1 time in total |
|
| Back to top |
|
|
| Esa |
| Posted: Mon Mar 19, 2012 6:18 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| mb01mqbrkrs wrote: |
I'm guessing that I'm going to have to design some custom retry logic using a persistent store - its a shame that the default broker retry logic cant be customised (to include a retry time). Any thoughts about this would be high appreciated. |
You can implement delayed retry logic with timer nodes. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Mar 19, 2012 6:19 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Esa wrote: |
| You can implement delayed retry logic with timer nodes. |
One of the methods previous dicussed I alude to above.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
