ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Dequeued failed message. Propagating a message to the failur

Post new topic  Reply to topic
 Dequeued failed message. Propagating a message to the failur « View previous topic :: View next topic » 
Author Message
mqxplorer
PostPosted: Tue Oct 02, 2012 1:28 pm    Post subject: Dequeued failed message. Propagating a message to the failur Reply with quote

Master

Joined: 22 Jun 2009
Posts: 206

Hi All,

As my latest other post, this is also a familiar issue, however, I am not able to find out the root cause. I have searched in this forum and went through a lot of posts...but still searcging .....

The exception message is this...

Code:
Dequeued failed message. Propagating a message to the failure terminal. .


My flow is a very simple flow. It reads a message from a queue, transforms it as per the web servcie schema format and calls the web service.

My MQInput node both terminals (try and catch) are connected to our common exception handler sub flow. As per my understadning of error handling for MQInput node, if there is an exception in the main flow, it rolls back to the catch terminal and if there is still an exception in the catch path, the original exception list will be over written and new exception list will be generated once the messages goes to the failure terminal. In my case, there is a very very less chance for the exception in the catch path. We are using the audit feature to generate the exception list using the xpath $ExceptionList on the MQOut node configured with an error queue. We capture the exeption list using the auidt flow which does a database insertion of the exception list. I could not really see any potential issue of failure in the cath path as error handler subflow simply generates the exception list as an audit event which will be takedn by the audit flow and the rolled back message will be put in the error queue. I have asked my admin to see if something is written in broekr logs and he says nothing is written there... As this is in Production and is happening once in a while I am not able to troubleshoot it in right way to find out the roort cause.

Can anybody help me suggesting some troubleshooting tips. Any help would be really appreciated..

Thanks
mqxplorer
Back to top
View user's profile Send private message
smdavies99
PostPosted: Tue Oct 02, 2012 7:05 pm    Post subject: Re: Dequeued failed message. Propagating a message to the fa Reply with quote

Jedi Council

Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.

mqxplorer wrote:


My MQInput node both terminals (try and catch) are connected to our common exception handler sub flow.



you must be using a different MQInput node to me. Mine don't have a 'try' terminal.

That aside, have you implemented a BOQ for the Input? If you do that properly then any failing message will be saved for you (i.e written to the BOQ) whereupon you can examine it, remove it to another system for testing etc etc etc.
If the errant message is ok then it might be that you have a resource issue downstream of the flow. It could also be that your common error handler is not getting all the relevant data for this particular exception and you have discovered a bug in it.
If the error of so infrequent then perhaps you need to deploy a 'special' version of the flow with a trace node in the Catch Branch and write the ExceptionList to a file.

I have always found that even though I might get the right error code 99% of the time in error handlers it always pays to save the complete exceptionList somewhere as well. doing so really does help track down issues with the exception handler itself.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995

Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions.
Back to top
View user's profile Send private message
mqxplorer
PostPosted: Wed Oct 03, 2012 11:22 am    Post subject: Reply with quote

Master

Joined: 22 Jun 2009
Posts: 206

smdavies99 wrote:


you must be using a different MQInput node to me. Mine don't have a 'try' terminal.


Sorry for the wrong terminology usage.... I meant to say Catch and failure terminals are connected to the common error handler sub flow.

After looking at our broker logs we found one new thing....

the broker log says...

Code:
Oct  2 10:34:23 XXXXX WebSphere Broker v7003[9159]: (BRK01.EG1)[7]BIP2643W: Unable to commit a WebSphere MQ transaction; MQCMIT failed; queue manager=BRK01, MQCC=1, MQRC=2003. : BRK01.4a03c68b-2701-0000-0080-ec999294ca75: /cmvc/mqsi/output/S700/amd64_linux_2/S700-FP03_P/src/DataFlowEngine/ImbMqManager.cpp: 2417: ImbMqConnection::commitCurrentTransaction: ComIbmMQConnectionManager: ComIbmMQConnectionManager


By looking at the info in IBM info center we got some Resolving the problem solution options...

http://www-01.ibm.com/support/docview.wss?uid=swg21039930

Code:
Resolving the problem
Use a shorter wait interval in your application. Issue MQCMIT on a more regular and frequent basis. Increase the number and size of you log files.


As explained in my post, my flow starts with an MQInput node, transforms the message to web service format and calls the web service. We have one abnormal thing (which I feel) here in our flow.. that is..due to the slow rpocessing of the web service request recievd by the web service, the reuest time out interval on the SOAP Request node is set to 600 seconds. The developer who developed this flows has long gone and Iam not sure what was the rationale in keeping this interval that high. I suspect that this is accounting to the first point of the "Resolving the Problem" from above. What would be the reasoble wait time interval for a web service call in broker flows.

based on the no of messages landing in the error queue, I got another question.. As I said, the catch and failure terminals of the MQInput node are connected to the common error handler subflow which is confgured with the error queue. I see two messages (same rolled back message) in the error queue. This means there is some exception the in the main flow which made the message go to catch terminal and message lands in the error queue. This is perfect.. However, I am confused about the rolled back message landing in the error queue again.. As per my understanding of error handling with MQInput node, when there is any exception in the catch path the message goes to failure terminal by overwiting the exception list created in the catch path. In my case, the error handler sub flow does not do much except writing the message to the error queue (which was successful in the catch path) and it just emits the monitoring event for the exceptionlist configured on the MQOutput node configured with error queue. The interesting thing here is.. if the message did go to the catch terminal of the MQInput node, I would expect the message landing in the error queue (which happened) and a monitoring event generated for the exceptionlist (which did not happen). I wonder what other possibilities are there for the message to land in the error queue twice...

As per your suggestion, I am going to deploy another evrsion of the flow with trace nodes in catch and failure paths of the MQInput node.

Thanks
mqxplorer
Back to top
View user's profile Send private message
mqsiuser
PostPosted: Wed Oct 03, 2012 7:37 pm    Post subject: Reply with quote

Yatiri

Joined: 15 Apr 2008
Posts: 637
Location: Germany

rekarm01 wrote:

failure-terminal: Exception WITHIN the node
catch: Downstream exception occured

Esa has also good posts about this and you (OP) pls get your terminology right: it is failure & catch (not catch & try).

Your problem is clear to me: Your transaction is big. Either reduce it (the 600 secs & msg-size) or increase the MQ-transaction log-size.
Back to top
View user's profile Send private message
rekarm01
PostPosted: Wed Oct 03, 2012 9:57 pm    Post subject: Re: Dequeued failed message. Propagating a message to the fa Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 1415

mqxplorer wrote:
As per my understanding of error handling for MQInput node, if there is an exception in the main flow, it rolls back to the catch terminal and if there is still an exception in the catch path, the original exception list will be over written and new exception list will be generated once the messages goes to the failure terminal.

That's not quite what happens. If an exception occurs in the catch branch, the current transaction terminates, it's exception list (along with everything else) gets discarded, and the message gets backed out to the input queue. The MQInput node then re-reads the input message, starting a new transaction. The MQInput node can then either propagate the message through its Out terminal (likely encountering the same exceptions in the Out and Catch branches again), or through its Failure terminal, (likely due to message backout threshold exceeded); in the event of failure, the exception list contains only the exceptions related to the failure itself.

Assuming the error is reproducible in a test environment, turning on a debug-level usertrace is the most effective way to determine what the message flow is actually doing. Adding trace nodes can help too.
Back to top
View user's profile Send private message
mqsiuser
PostPosted: Thu Oct 04, 2012 12:46 am    Post subject: Re: Dequeued failed message. Propagating a message to the fa Reply with quote

Yatiri

Joined: 15 Apr 2008
Posts: 637
Location: Germany

The error message says the following

Code:
Dequeued failed message.

The MQ Input Node gets a message from a queue (de-queues a msg). And this failed... Probably there is no / not enough space for starting a new transaction ?

Code:
Propagating a message to the failure terminal.

An internal (internal to the MQ Input Node) error occured and a msg is send to the failure-terminal.

What does this msg look like ?
Back to top
View user's profile Send private message
McueMart
PostPosted: Thu Oct 04, 2012 1:03 am    Post subject: Reply with quote

Chevalier

Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere

If you are using a common error handling subflow, and are connecting both the Catch and Failure terminals to it, it is worth checking the MQMD.BackoutCount. If this is >= 1, this means that the message has already probably already been down your MQInput -> Out -> *Exception* --> Catch path, so you dont want to process this same error again.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Dequeued failed message. Propagating a message to the failur
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.