| Author |
Message
|
| angka |
 Posted: Tue Oct 27, 2009 2:24 am Post subject: Exception when retrying a HTTPRequest |
 |
|
Chevalier
Joined: 20 Sep 2005
Posts: 406
|
Below is my flow which cause error:
MQInput-out->ComputeURL-out->HTTPRequest-failure->ComputeFailure-out->ComputeURL
When the web server is down, it will route through the failure connection to the ComputeFailure node. in the ComputeFailure node, it did a thread.sleep and it routes back to the ComputeURL node. Then the error occur when it routes to the HTTPRequest node. Can't catch any exception. the execution grp restarted.
I did a user trace and below is the error:
Timestamps are formatted in local time, 480 minutes past GMT.
Trace written by version 6105; formatter version 6105
2009-10-27 17:57:09.227129 3776 UserTrace BIP2632I: Message received and propagated to 'out' terminal of MQ input node 'HttpRequest.MQInput'.
2009-10-27 17:57:09.227521 3776 UserTrace BIP6060I: Parser type ''Properties'' created on behalf of node 'HttpRequest.MQInput' to handle portion of incoming message of length 0 bytes beginning at offset '0'.
2009-10-27 17:57:09.227638 3776 UserTrace BIP6061I: Parser type ''MQMD'' created on behalf of node 'HttpRequest.MQInput' to handle portion of incoming message of length '364' bytes beginning at offset '0'. Parser type selected based on value ''MQHMD'' from previous parser.
2009-10-27 17:57:09.227711 3776 UserTrace BIP6061I: Parser type ''XMLNSC'' created on behalf of node 'HttpRequest.MQInput' to handle portion of incoming message of length '751' bytes beginning at offset '364'. Parser type selected based on value ''XMLNSC'' from previous parser.
2009-10-27 17:57:09.227944 3776 UserTrace BIP4007I: Message propagated to 'out' terminal of node 'HttpRequest.Compute'.
2009-10-27 17:57:09.232863 3776 UserTrace BIP4007I: Message propagated to 'out' terminal of node 'HttpRequest.ComputeURL'.
2009-10-27 17:57:15.244553 3776 UserTrace BIP2231E: Error detected whilst processing a message in node 'HttpRequest.HTTP Request'.
The message broker detected an error whilst processing a message in node 'HttpRequest.HTTP Request'. The message has been augmented with an exception list and has been propagated to the node's failure terminal for further processing.
See the following messages for details of the error.
2009-10-27 17:57:15.244582 3776 RecoverableException BIP3152S: Socket error detected whilst invoking Web service located at host '10.10.10.11', port 7080, path '/test/webservice'.
This may be a transient error, such as a server not responding, or a symptom of an invalid hostname or port number.
Check that the hostname and port number are valid, and point to a functioning Web service.
2009-10-27 17:57:15.244592 3776 SocketTimeoutException BIP3151S: A socket operation timed out. Operation: '::select()'. Error Code: 0. Timeout in seconds: 6. Error Text: 'The operation completed successfully.
'.
A timeout occurred whilst performing a TCP/IP socket operation. This may be a transient error, such as a server is busy or it could be that the server is no longer responding.
See the following messages for information pertaining to this error.
2009-10-27 17:57:15.245174 3776 UserTrace BIP4007I: Message propagated to 'out' terminal of node 'HttpRequest.ComputeFailure'.
2009-10-27 17:57:15.245353 3776 UserTrace BIP4007I: Message propagated to 'out' terminal of node 'HttpRequest.ComputeURL'.
2009-10-27 17:57:19.259433 2868 Information BIP7033I: Publish/Subscribe state recovery completed.
Recovery of the Publish/Subscribe state from the broker's database is complete.
This message is for information only. No action is required.
2009-10-27 17:57:24.153493 3720 Error BIP2648E: Message backed out to a queue; node 'HttpRequest.MQInput'.
Node 'HttpRequest.MQInput' has received a message which has previously been backed out one or more times because of a processing error in the main path of the message flow. The failure terminal is not attached, so the message broker is putting the message directly to the requeue or dead letter backout queue associated with this node. The MQMD 'backoutCount' of the message now equals the 'backoutThreshold' defined for the WebSphere MQ input queue.
Examine previous messages and the message flow to determine why the message is being backed out. Correct this situation if possible. Perform any local error recovery processing required.
2009-10-27 17:57:24.154029 3720 UserTrace BIP2638I: The MQ output node 'HttpRequest.MQInput' attempted to write a message to queue ''SYSTEM.DEAD.LETTER.QUEUE'' connected to queue manager ''IKC2_QM1''. The MQCC was '0' and the MQRC was '0'.
2009-10-27 17:57:24.154048 3720 UserTrace BIP2615I: The WebSphere MQ input node 'HttpRequest.MQInput' has backed out the message to the backout requeue or the dead letter queue.
Message backout processing has been invoked, and the message has either been backed out by being written to the backout requeue or dead letter queue, as determined by the WebSphere MQ queue manager and queue configuration.
No user action required.
Threads encountered in this trace:
2868 3720 3776
Thanks |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Tue Oct 27, 2009 2:34 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| If you want to Catch an Exception then don't use the failure terminal. |
|
| Back to top |
|
|
| angka |
| Posted: Tue Oct 27, 2009 2:43 am Post subject: |
|
|
Chevalier
Joined: 20 Sep 2005
Posts: 406
|
Hi,
I tested with HttpInput node. when i stop the HTTPinput flow in another server, it will route to the failure connection. i need to handle the failure exception also. This shld be quite simple.
btw when the webserver reply with http error 500, it routes to the error connection and i will try to call the HTTPRequest node again after thread sleep.. the same thing happen.
Thanks |
|
| Back to top |
|
|
| angka |
| Posted: Tue Oct 27, 2009 3:47 pm Post subject: |
|
|
Chevalier
Joined: 20 Sep 2005
Posts: 406
|
Hi,
Anyone can help??
Thanks |
|
| Back to top |
|
|
| sridhsri |
| Posted: Wed Oct 28, 2009 12:27 pm Post subject: |
|
|
Master
Joined: 19 Jun 2008
Posts: 297
|
| As mqjeff said, try to wire the catch terminal instead of the failure and perform your error processing there. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Oct 28, 2009 12:35 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Or put a Try/Catch node before the request node.
You know, like you were writing a Try/Catch block in any other language that supports exceptions. |
|
| Back to top |
|
|
| angka |
| Posted: Wed Nov 04, 2009 7:17 pm Post subject: |
|
|
Chevalier
Joined: 20 Sep 2005
Posts: 406
|
Hi,
The problem is solved when i increase the environment variable MQSI_THREAD_STACK_SIZE. However this may not solve my problem because eventually if I keep looping, it will hit the value I set.
Thanks. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Thu Nov 05, 2009 2:26 am Post subject: Re: Exception when retrying a HTTPRequest |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| angka wrote: |
| The problem is solved when i increase the environment variable MQSI_THREAD_STACK_SIZE. However this may not solve my problem because eventually if I keep looping, it will hit the value I set. |
If there's an actual node loop in the message flow, increasing the MQSI_THREAD_STACK_SIZE won't resolve the problem:
| Code: |
MQInput --> Compute --> HTTPRequest --+--> (out) --> ...
^ +--> (failure)
| |
+--<--<--(Thread.sleep)--<--<--+ |
This message flow is vulnerable to stack overflow, if the web service is down for too long. It also won't help to implement such a loop with either MQInput.catch or a TryCatch node, rather than HTTPRequest.failure.
The only way to implement retry logic inside the message flow is to completely back out the message, (by throwing an uncaught exception). In that case, it will probably also be necessary to increase the backout threshold for the input queue, or define a backout queue.
A much better approach is to notify the sending application that the transaction failed, and have it implement any delay/retry logic, outside of the broker. |
|
| Back to top |
|
|
| angka |
| Posted: Thu Nov 05, 2009 11:30 pm Post subject: |
|
|
Chevalier
Joined: 20 Sep 2005
Posts: 406
|
Hi,
Ya i changed my design already.. due to message affinity issue i can't put the message to a backout queue. I put it back to the input queue with priority set and use a thread.sleep to retry. i will increase my sleep interval with each retry using shared variable. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Nov 06, 2009 3:19 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| angka wrote: |
Hi,
Ya i changed my design already.. due to message affinity issue i can't put the message to a backout queue. I put it back to the input queue with priority set and use a thread.sleep to retry. i will increase my sleep interval with each retry using shared variable. |
Bad design. Message affinity will kill you.
What are you going to do if the web service is down for 3 days? or a week?
Eventually you will need to get inhibit the queue and stop the flow...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
