ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » MQGet node not performing backout?

Post new topic  Reply to topic Goto page 1, 2  Next
 MQGet node not performing backout? « View previous topic :: View next topic » 
Author Message
mb01mqbrkrs
PostPosted: Mon Mar 19, 2012 3:28 am    Post subject: MQGet node not performing backout? Reply with quote

Apprentice

Joined: 18 Nov 2011
Posts: 48

Hi Experts.
I've been performing some testing with the MQGet node, but seem to have a problem with the backout handling...

HTTP Input (just to trigger the flow) -> MQGet-> Throw

Each time the flow is run, the message on the queue that MQGet is configured to use, has its BOCount increased by one - which is what I would expect.

However, the BOCount seems to be able to increase over the BOThreshold for the queue and the message doesn't hit the BOQ.

Any ideas? - or is this by design?

MB 6.1.0.7
Back to top
View user's profile Send private message
Esa
PostPosted: Mon Mar 19, 2012 3:43 am    Post subject: Re: MQGet node not performing backout? Reply with quote

Grand Master

Joined: 22 May 2008
Posts: 1387
Location: Finland

How much over the treshold?

I am not sure, but I think MQInput puts the message in BOQ after getting the message, if it detects that backout count exeeds the treshold. MQGet should do the same. So the message may not end up in the backout queue when the transaction is backed out, because no MQGet node code is not executed when the backout happens. It should happen the next time you try to MQget.

But it's possible that this functionality has not been implemented in MQget node.
Back to top
View user's profile Send private message
mb01mqbrkrs
PostPosted: Mon Mar 19, 2012 3:47 am    Post subject: Reply with quote

Apprentice

Joined: 18 Nov 2011
Posts: 48

Thanks for the response Esa. To answer your question, the backout count in my testing has increased to more than double that of the backout threshold.
Back to top
View user's profile Send private message
Vitor
PostPosted: Mon Mar 19, 2012 4:59 am    Post subject: Re: MQGet node not performing backout? Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

mb01mqbrkrs wrote:
Any ideas? - or is this by design?


What's the transactionality of your flow?

I'd theorise that the message is not places in the backout queue because it's not the driving message of the flow and needs to be in the same queue if the principle flow restarts. Remember the MQGet node is designed to be used to read a specific message from a queue as part of flow processing; it's not intended to read & empty a queue. So it's reasonable to assume that you'd want that specific message to remain on the original queue.

I'd be interested in IBM's opinion on this via a PMR. Please let us know what they say.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Esa
PostPosted: Mon Mar 19, 2012 5:10 am    Post subject: Re: MQGet node not performing backout? Reply with quote

Grand Master

Joined: 22 May 2008
Posts: 1387
Location: Finland

Precisely.

Why is the driving message written to a backout queue? To be able to resend it manually after the error situation has been corrected, for example. If the side message would be written to a backout queue too, the flow would fail for the resent message.

It is quite usual that messages in the MQGet queue have an expriration set. That will take care of queue cleanup.
Back to top
View user's profile Send private message
mb01mqbrkrs
PostPosted: Mon Mar 19, 2012 5:32 am    Post subject: Re: MQGet node not performing backout? Reply with quote

Apprentice

Joined: 18 Nov 2011
Posts: 48

Thanks for the replies.

The example flow was just a proof of concept to see if it was possible to trigger reprocessing of messages on an error queue (basically allowing retries with a timer gap). Obviously we wouldn't want the same poison message retried infinitely, which is why I was looking at the backout behaviour of the MQGet node.

Vitor wrote:
Remember the MQGet node is designed to be used to read a specific message from a queue as part of flow processing; it's not intended to read & empty a queue.

Interesting - I'm sure that your right, but I cant see any reference to this in the product documentation and I cant remember this being said in the training course.
Back to top
View user's profile Send private message
Vitor
PostPosted: Mon Mar 19, 2012 5:37 am    Post subject: Re: MQGet node not performing backout? Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

mb01mqbrkrs wrote:
Vitor wrote:
Remember the MQGet node is designed to be used to read a specific message from a queue as part of flow processing; it's not intended to read & empty a queue.

Interesting - I'm sure that your right, but I cant see any reference to this in the product documentation and I cant remember this being said in the training course.


The product documentation is focused on what a feature can do, rather than how to use it. I'd have expected some mention of this in the design section of a training course.

You'll find a wealth of discussion in here on the perils of repeatedly calling an MQGet node (i.e. looping round to empty an error queue). Mostly people running out of memory and/or seeing execution groups fail.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Vitor
PostPosted: Mon Mar 19, 2012 5:41 am    Post subject: Re: MQGet node not performing backout? Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

mb01mqbrkrs wrote:
The example flow was just a proof of concept to see if it was possible to trigger reprocessing of messages on an error queue (basically allowing retries with a timer gap).


There are a number of discussions in here on various ways of achiving this.

mb01mqbrkrs wrote:
Obviously we wouldn't want the same poison message retried infinitely


IMHO you wouldn't want it to be retried automatically more than once. If a transient problem is still a problem when you retry, you either have the retry interval set too short or the problem isn't transient!

You also need to factor in such intangibles as how long the business will tollerate the message whizzing round and round your automatic retry, how this affects the life of your message, etc, etc.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Vitor
PostPosted: Mon Mar 19, 2012 5:44 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

You also need to bear in mind that this kind of record & replay technology is built in to v8, so if this is a key site requirement you might want to consider going to WMBv8 rather than WMBv7.

Because clearly you're not going to be on WMBv6.1.0.7 much longer. In an ideal world you wouldn't be on it now (you'd be on v6.1.0.10).
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
mb01mqbrkrs
PostPosted: Mon Mar 19, 2012 5:59 am    Post subject: Reply with quote

Apprentice

Joined: 18 Nov 2011
Posts: 48

Vitor wrote:

IMHO you wouldn't want it to be retried automatically more than once. If a transient problem is still a problem when you retry, you either have the retry interval set too short or the problem isn't transient!

You also need to factor in such intangibles as how long the business will tollerate the message whizzing round and round your automatic retry, how this affects the life of your message, etc, etc.


Unfortunately in this particular case, the message could fail going to our ERP system due to the record being locked. The record could be unlocked "sometime" in the future (this could be hours). Clearly, if I could just use the built-in Broker retry logic I would - but I needed some way of retrying every hour or so.
I understand that the requirements are far less than ideal and have pushed back without success - apparently this functionality is standard from our ERP Vendor!

I'm guessing that I'm going to have to design some custom retry logic using a persistent store - its a shame that the default broker retry logic cant be customised (to include a retry time). Any thoughts about this would be high appreciated.
Back to top
View user's profile Send private message
Esa
PostPosted: Mon Mar 19, 2012 6:02 am    Post subject: Re: MQGet node not performing backout? Reply with quote

Grand Master

Joined: 22 May 2008
Posts: 1387
Location: Finland

mb01mqbrkrs wrote:
Thanks for the replies.

The example flow was just a proof of concept to see if it was possible to trigger reprocessing of messages on an error queue (basically allowing retries with a timer gap). Obviously we wouldn't want the same poison message retried infinitely, which is why I was looking at the backout behaviour of the MQGet node.


You can isolate processing of the MQGet message (with a TryCatch node, for example) and after an exception examine backout count programatically (against a constant or UDP). Because your flow is transactional, you must put a copy of the poison MQGet message in another queue if the count is exeeded. Then have another flow read it, remove the message with matching id from the MQGet queue and write it to "poison message" queue.

If I understood correctly what you were trying to do.
Back to top
View user's profile Send private message
mqsiuser
PostPosted: Mon Mar 19, 2012 6:04 am    Post subject: Reply with quote

Yatiri

Joined: 15 Apr 2008
Posts: 637
Location: Germany

So there are these properties. And then your ERP-System will read them somehow: Either MQ or JMS (from a queue)... the properties can be retrieved, e.g. by your ERP-Software-Vendor "JMS-Driver"... and this driver can be configured to do "custom" logic based on the "properties"... properties... only some values (no logic from MQ involved!)... The logic is defined by the reading application (your JMS-Driver from your ERP-System).
_________________
Just use REFERENCEs


Last edited by mqsiuser on Mon Mar 19, 2012 6:18 am; edited 2 times in total
Back to top
View user's profile Send private message
Vitor
PostPosted: Mon Mar 19, 2012 6:08 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

mb01mqbrkrs wrote:
Unfortunately in this particular case, the message could fail going to our ERP system due to the record being locked. The record could be unlocked "sometime" in the future (this could be hours).


So the user community are happy for records to be sitting happy and unprocessed for hours (which can easily turn into days)? What happens when you finally hit a limit of some kind and the message is deemed "poisoned"? Most users of my experience would react along the lines of "the ERP has been locked up for 12 hours? Nothing's going through? Where's the trouble ticket? What do you mean there's no trouble ticket until the automatic retry expires???"

mb01mqbrkrs wrote:
I understand that the requirements are far less than ideal and have pushed back without success - apparently this functionality is standard from our ERP Vendor!


You need a better ERP vendor.

mb01mqbrkrs wrote:
I'm guessing that I'm going to have to design some custom retry logic using a persistent store


See my comments above; you may find some of the discussion helpful.

mb01mqbrkrs wrote:
its a shame that the default broker retry logic cant be customised (to include a retry time).


See my comments above re WMBv8.
_________________
Honesty is the best policy.
Insanity is the best defence.


Last edited by Vitor on Mon Mar 19, 2012 6:18 am; edited 1 time in total
Back to top
View user's profile Send private message
Esa
PostPosted: Mon Mar 19, 2012 6:18 am    Post subject: Reply with quote

Grand Master

Joined: 22 May 2008
Posts: 1387
Location: Finland

mb01mqbrkrs wrote:

I'm guessing that I'm going to have to design some custom retry logic using a persistent store - its a shame that the default broker retry logic cant be customised (to include a retry time). Any thoughts about this would be high appreciated.


You can implement delayed retry logic with timer nodes.
Back to top
View user's profile Send private message
Vitor
PostPosted: Mon Mar 19, 2012 6:19 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

Esa wrote:
You can implement delayed retry logic with timer nodes.




One of the methods previous dicussed I alude to above.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » MQGet node not performing backout?
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.