| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| Environment Tree Getting Cleared |
« View previous topic :: View next topic » |
| Author |
Message
|
| sarathmattam |
 Posted: Wed May 08, 2013 11:27 am Post subject: Environment Tree Getting Cleared |
 |
|
Voyager
Joined: 05 Sep 2008
Posts: 94
|
Dear All,
I am facing an issue while handling an error case. My Message flow is as below
MQ Input -- > Compute Node --> Soap Request Node --> MQ OutPut
I will receive a message in a Queue. Based on the content of the message, I will populate the message and trigger an webservice. The response will be sent back to the source.
Now, the issue is that i have to handle errors and retry if its a socket exception. I am setting the retry count in an Environment variable. when the exception is thrown, the message is rolled back to the MQ input node. The retry flow (which is a sub flow) is attached to both catch and failure terminal of MQ Input node. But, the messge is propagated to the Failure terminal (instead of Catch terminal). Also the Environment Variables are getting cleared. Because of this my retry scenario is failing ..
My queries are
1. Why message is going to the Failure Terminal. I hope it should go to Catch terminal?
2. What could be the reason for environment variables to get cleared? |
|
| Back to top |
|
 |
| Vitor |
| Posted: Wed May 08, 2013 12:07 pm Post subject: Re: Environment Tree Getting Cleared |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sarathmattam wrote: |
| 1. Why message is going to the Failure Terminal. I hope it should go to Catch terminal? |
Are you sure? Does a user trace show the message going through the Failure terminal? What difference does it make which terminal you went through to get to your handling?
| sarathmattam wrote: |
| 2. What could be the reason for environment variables to get cleared? |
Because that's what happens. See here
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Wed May 08, 2013 6:07 pm Post subject: Re: Environment Tree Getting Cleared |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| sarathmattam wrote: |
| The retry flow (which is a sub flow) is attached to both catch and failure terminal of MQ Input node. |
Tying both the Catch and Failure terminal to the same error handling subflow is a common practice, but the two terminals serve different purposes, so it doesn't always make sense to do this.
| sarathmattam wrote: |
| when the exception is thrown, the message is rolled back to the MQ input node. |
Environment trees (and Exceptions) do not persist from one transaction to the next. When a message flow backs out a message to its input queue, that terminates the transaction. Any subsequent retry starts a new transaction.
The MQMD.BackoutCount field is better suited for counting the number of retries. But if the BackoutCount exceed's the queue's Backout Threshold, that would cause the message flow to route messages to the MQInput Failure terminal. |
|
| Back to top |
|
|
| kash3338 |
| Posted: Thu May 09, 2013 12:30 am Post subject: Re: Environment Tree Getting Cleared |
|
|
Shaman
Joined: 08 Feb 2009
Posts: 709
Location: Chennai, India
|
| sarathmattam wrote: |
1. Why message is going to the Failure Terminal. I hope it should go to Catch terminal?
|
Are you sure the message has hit the service and then rolled back? Could it be a case of failing in the MQInput Node itself? Are you validating the incoming message in your input node? What does the user trace say?
| sarathmattam wrote: |
2. What could be the reason for environment variables to get cleared? |
This works as documented. Pointed out by Vitor earlier. |
|
| Back to top |
|
|
| mapa |
| Posted: Thu May 09, 2013 1:26 am Post subject: |
|
|
Master
Joined: 09 Aug 2001
Posts: 257
Location: Malmö, Sweden
|
I think the OP is trying to use the catch to do retries before backing out the message, could it be so?
Probably due to using SOAP over HTTP/S which might not always be reliable, and basically the broker has no built in functionality for retries (which kind of sucks).
If so, LocalEnvironment and message gets cleared, Environment is intact until the flow is completed.
However if it is propagated to the failure terminal of MQInput instead of catch then it is probably a parsing or conversion error. This is what others have suggested as well, that the error occurrs already in the input node.
Provide more information of input message, mqinput node config.
Or even better, a usertrace... |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu May 09, 2013 3:03 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| mapa wrote: |
| Probably due to using SOAP over HTTP/S which might not always be reliable, and basically the broker has no built in functionality for retries (which kind of sucks). |
Can't let this fly by without a challenge.
>> might not always be reliable
The protocol is very reliable, the server might not be.
>> basically
No, not basically.
>> the broker has no built in functionality for retries
Thats not true. You can catch and propagate as many times as you want. Whats wrong with that?
>> (which kind of sucks)
The broker runtime will do exactly what you tell it do to. If you want it to behave in a repetitive manner, all the developer needs to do is code it so.
Jean-Luc Piccard : "MAKE IT SO".
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| mapa |
| Posted: Thu May 09, 2013 3:30 am Post subject: |
|
|
Master
Joined: 09 Aug 2001
Posts: 257
Location: Malmö, Sweden
|
| lancelotlinc wrote: |
Can't let this fly by without a challenge.
>> might not always be reliable
The protocol is very reliable, the server might not be.
|
I was talking about services and they can temporarily fail for a number of reasons, like a server restart or unstable network, using HTTP the client is responsible for handling errors. One of the reasons why REST has become so popular since it uses the distributed model of the Internet, where things fail temporarily quite often...So I got a 500 instead of a 201, try again, wow, now it worked 1 sec later. (There are a lot of other reasons of course.)
| lancelotlinc wrote: |
>> basically
No, not basically.
|
Might be my English? Skip the word basically from the sentence.
| lancelotlinc wrote: |
>> the broker has no built in functionality for retries
Thats not true. You can catch and propagate as many times as you want. Whats wrong with that?
|
My point exactly, no retry functionality built in. (but support for building solutions with retry using built in functionality). There is no option on the HTTP or SOAP request node where you can say, please do 5 short retries and then 2 long ones for HTTP error codes 5xx (but fail on 4xx for instance). Take a look at how other products/frameworks handles this and you will find that it is not uncommon to have much better support for this.
| lancelotlinc wrote: |
>> (which kind of sucks)
The broker runtime will do exactly what you tell it do to. If you want it to behave in a repetitive manner, all the developer needs to do is code it so.
|
Yes, of course, what the developer code it to do, instead of configure it to do.
My point exactly once again.
Adding a retry node (like a built in try-catch with x number of retries, which is not hard to build for an experienced WMB developer, still you have to do it yourself) could be one option |
|
| Back to top |
|
|
| goffinf |
| Posted: Thu May 09, 2013 8:14 am Post subject: |
|
|
Chevalier
Joined: 05 Nov 2005
Posts: 401
|
| lancelotlinc wrote: |
The protocol is very reliable, the server might not be.
|
Well possibly at the TCP 'handshake' level but not in terms of other desirable QoS factors.
It might be OK for 'safe' or 'idempotent' exchanges and those that are 'single hop' and use a simple synchronous request/reply pattern, but after that you're (a) taking a big [business] risk and (b) giving yourself a lot of work to do to make it reliable. If that's the majority of your world, then that's great, the implementation style is certainly the easiest. It doesn't reflect the integration solutions that I mostly work on though .
First chance I get I come off HTTP and mediate on to something that provides the QoS most business really need as basic characteristics of the infrastructure as well as providing a host of other run-time properties. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu May 09, 2013 9:11 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| goffinf wrote: |
| lancelotlinc wrote: |
The protocol is very reliable, the server might not be.
|
Well possibly at the TCP 'handshake' level but not in terms of other desirable QoS factors.
It might be OK for 'safe' or 'idempotent' exchanges and those that are 'single hop' and use a simple synchronous request/reply pattern, but after that you're (a) taking a big [business] risk and (b) giving yourself a lot of work to do to make it reliable. If that's the majority of your world, then that's great, the implementation style is certainly the easiest. It doesn't reflect the integration solutions that I mostly work on though .
First chance I get I come off HTTP and mediate on to something that provides the QoS most business really need as basic characteristics of the infrastructure as well as providing a host of other run-time properties. |
This statement and the attitude that SOAP is somehow deficient more so than any other protocol is incredulous.
I don't know a bank or insurance company that doesn't use SOAP. Or a kid in the shower for that matter. Well, ok, maybe some kids don't.
SOAP is as reliable or QoS-ified as you code it to be. If you want to retry three times, do it. Don't complain about how the Broker won't do it automatically for you. If your SOAPInput node has a timeout set for 7 seconds, and your happy-path SOAPRequest takes three seconds, you wouldn't want to retry that SOAPRequest automatically, would you?
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| mapa |
| Posted: Thu May 09, 2013 9:34 am Post subject: |
|
|
Master
Joined: 09 Aug 2001
Posts: 257
Location: Malmö, Sweden
|
Of course in your scenario, no I wouldn't, and yes, once again, it is not very hard to do as you also state. Still there is no default concept of retry built into the broker for this and if you search this forum you will see that people has(have) struggled with getting this right more than once.
Maybe a misunderstanding, I was not talking about SOAP which is transport protocol neutral, (although I think that still the vast majority is using HTTP/S.)
I am talking about HTTP/S which is by design - unreliable, but has excellent support for using it as an application protocol (as the REST-afarians have re-discovered).
Arguably you can use WS-RM, but that is possible first in WMB8.
I've never used this myself so far and there are other voices saying you shouldn't need it and I am inclined to agree. I would make sure to use "proper" messaging like WMQ instead if possible.
Nobody Needs Reliable Messaging
(IBM even tried(perhaps even did) develop the HTTPR protocol for reliable transactions, not sure it got any traction.) |
|
| Back to top |
|
|
| goffinf |
| Posted: Thu May 09, 2013 10:18 am Post subject: |
|
|
Chevalier
Joined: 05 Nov 2005
Posts: 401
|
| lancelotlinc wrote: |
| goffinf wrote: |
| lancelotlinc wrote: |
The protocol is very reliable, the server might not be.
|
Well possibly at the TCP 'handshake' level but not in terms of other desirable QoS factors.
It might be OK for 'safe' or 'idempotent' exchanges and those that are 'single hop' and use a simple synchronous request/reply pattern, but after that you're (a) taking a big [business] risk and (b) giving yourself a lot of work to do to make it reliable. If that's the majority of your world, then that's great, the implementation style is certainly the easiest. It doesn't reflect the integration solutions that I mostly work on though .
First chance I get I come off HTTP and mediate on to something that provides the QoS most business really need as basic characteristics of the infrastructure as well as providing a host of other run-time properties. |
This statement and the attitude that SOAP is somehow deficient more so than any other protocol is incredulous.
I don't know a bank or insurance company that doesn't use SOAP. Or a kid in the shower for that matter. Well, ok, maybe some kids don't.
SOAP is as reliable or QoS-ified as you code it to be. If you want to retry three times, do it. Don't complain about how the Broker won't do it automatically for you. If your SOAPInput node has a timeout set for 7 seconds, and your happy-path SOAPRequest takes three seconds, you wouldn't want to retry that SOAPRequest automatically, would you? |
You need to separate your concerns here.
We're not really talking about SOAP, or at least I'm not.
SOAP has a range of possible transport bindings that have different characteristics that provide some of the reliability and other QoS attributes. Not everything, agreed, but choosing the binding is often driven by such concerns.
SOAP by itself is a pretty lightweight messaging protocol and doesn't really provide anything by itself to support these concepts (it's not really much more than a container for other stuff) . Those other (layered or if you like 'composable') protocols (like WS-Addressing, WS-RM and the like) might have more to say about reliability, but of course then your into interop territory and the capabilities of both consumer and provider end points, some of whom won't be able to participate (not everyone uses Broker you know  and in my case (which I sure is fairly typical) we have lots of different 'stacks' attached to apps around our organisation and those of our trading partners.
Lancelotlinc, I know that you know this stuff (I've read enough of your posts to be sure about that), but, just like you, sometimes I find a 'lazy' comment too tempting to let go past.
The article that 'mapa' refers to was written some time ago and, as far as I can recall, talks quite a lot about trying to design your services to be idempotent and/or safe. Sadly, that is very often impossible (not only for technical reasons but often as not because of cost) other than in simple 'point to point' synchronous exchanges.
As far as retry capability, I agree that it is perfectly possible to create that from the general purpose tooling MB provides. However I also agree with 'mapa' that when an organisation commits to high level application development tooling it often does so in the expectation that common use cases like this should be available OOTB. You may have heard the expression 'why have a dog and bark yourself' !
Mapa , how about submitting an enhancement request ?
Regards
Fraser. |
|
| Back to top |
|
|
| mapa |
| Posted: Thu May 09, 2013 12:06 pm Post subject: |
|
|
Master
Joined: 09 Aug 2001
Posts: 257
Location: Malmö, Sweden
|
Yes, why not put a RFE, can't hurt, better than to keep complaining about it I guess 
All too often I end up being forced to invoke generic-SOAP services passing in the real payload XML as either CDATA or encoded XML in an element called "xml"...sigh.
Even in new projects with ERPs like MS Dynamics AX 2012 
I know all too well that there is a big difference between what can be done and what we often end up doing... |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Fri May 10, 2013 12:15 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| mapa wrote: |
I think the OP is trying to use the catch to do retries before backing out the message, could it be so?
...
If so, LocalEnvironment and message gets cleared, Environment is intact until the flow is completed. |
Backing out the message would complete the transaction, and clear the Environment. Any subsequent retries would occur in a separate transaction, and if the input queue's backout threshold is set to (default value) 0, then the retries would all fail in the MQInput node, and propagate to the MQInput.Failure terminal.
Consult the InfoCenter here, here, and here, for more details. |
|
| Back to top |
|
|
| mapa |
| Posted: Fri May 10, 2013 6:56 am Post subject: |
|
|
Master
Joined: 09 Aug 2001
Posts: 257
Location: Malmö, Sweden
|
| rekarm01 wrote: |
| mapa wrote: |
I think the OP is trying to use the catch to do retries before backing out the message, could it be so?
...
If so, LocalEnvironment and message gets cleared, Environment is intact until the flow is completed. |
Backing out the message would complete the transaction, and clear the Environment. Any subsequent retries would occur in a separate transaction, and if the input queue's backout threshold is set to (default value) 0, then the retries would all fail in the MQInput node, and propagate to the MQInput.Failure terminal.
|
Yes, and that is what I am saying. (Or at least tried to say).
That Environment is intact after the catch so that the Environment.variables could be used for getting information set in the flow before it fails. The OP is asking why it goes to failure instead of catch although the catch is wired. At least that is how I interpreted the question. We won't know unless we get a usertrace. |
|
| Back to top |
|
|
| sarathmattam |
| Posted: Sun May 12, 2013 12:55 am Post subject: |
|
|
Voyager
Joined: 05 Sep 2008
Posts: 94
|
Here is the User Trace ..
@ Mapa : your understanding is perfect and that is what am asking for. I am using environment variables to set some values to use in case of errors. Also i have wired the CATCH terminal of my MQ Input node .. But, instead its going to the FAILURE terminal.
2013-05-12 12:46:16.743614 36404 UserTrace BIP4015I: Message propagated to the 'out' terminal of node 'com.emaratech.vision.esb.event.outpass.VM_Outpass_Event.FindByOutpassNo' with the following message trees: ''.
2013-05-12 12:46:16.746437 36404 UserTrace BIP3719I: Node 'FindByOutpassNoRequest' sending SOAP request via transport 'HTTP' ('www.vision.ae:9080') for operation 'findByOutpassNo' of type 'request-response'.
See subsequent messages for success or failure messages relating to this request, and for any transport-specific messages.
No action required.
2013-05-12 12:46:16.752493 36404 UserTrace BIP3633I: Node 'FindByOutpassNoRequest' sending HTTP data to URL '/visioncore/OutpassService' at host 'www.vision.ae' (port 9080).
The broker is sending data via HTTP to a remote server at host 'www.vision.ae' (port 9080) using URL '/visioncore/OutpassService'. See subsequent messages for success or failure messages relating to this request.
No action required.
2013-05-12 12:46:21.358374 36404 UserTrace BIP3634I: Node 'FindByOutpassNoRequest' received HTTP data from host 'www.vision.ae' with status code of 404.
The broker has received data via HTTP from a remote server at host 'www.vision.ae' with an HTTP status code of 404. Status codes beginning with a 2 indicate success; others imply errors in either client or server.
No action required.
2013-05-12 12:46:21.361064 36404 Error BIP2628E: Exception condition detected on input node 'com.emaratech.vision.esb.event.outpass.VM_Outpass_Event.Outpass_EventIn'.
The input node 'com.emaratech.vision.esb.event.outpass.VM_Outpass_Event.Outpass_EventIn' detected an error whilst processing a message. The message flow has been rolled-back and, if the message was being processed in a unit of work, it will remain on the input queue to be processed again. Following messages will indicate the cause of this exception.
Check the error messages which follow to determine why the exception was generated, and take action as described by those messages.
2013-05-12 12:46:21.361102 36404 RecoverableException BIP2230E: Error detected whilst processing a message in node 'com.emaratech.vision.esb.event.outpass.VM_Outpass_Event.FindByOutpassNoRequest'.
The message broker detected an error whilst processing a message in node 'com.emaratech.vision.esb.event.outpass.VM_Outpass_Event.FindByOutpassNoRequest'. An exception has been thrown to cut short the processing of the message.
See the following messages for details of the error.
2013-05-12 12:46:21.361136 36404 RecoverableException BIP3754E: The SOAP Request Node or SOAP Async Request Node com.emaratech.vision.esb.event.outpass.VM_Outpass_Event.FindByOutpassNoRequest encountered an error while processing the outbound SOAP request.
An error occurred while processing the SOAP Request.
See previous error messages for an indication to the cause of the errors.
2013-05-12 12:46:21.361286 36404 RecoverableException BIP3162S: An HTTP error occurred. The HTTP Request-Line was: ''POST /visioncore/OutpassService HTTP/1.1
''.
The HTTP Request Header bitstream (if any) to be used was: 'X'436f6e74656e742d4c656e6774683a203330370d0a436f6e74656e742d547970653a20746578742f786d6c3b20636861727365743d7574662d380d0a486f73743a207777772e766973696f6e2e61653a393038300d0a534f4150416374696f6e3a202266696e6442794f7574706173734e6f220d0a436f6e6e656374696f6e3a204b6565702d416c6976650d0a0d0a''. The HTTP Request Message Body bitstream (if any) to be used was: 'X'3c3f786d6c2076657273696f6e3d22312e302220656e636f64696e673d227574662d38223f3e3c736f6170656e763a456e76656c6f706520786d6c6e733a736f6170656e763d22687474703a2f2f736368656d61732e786d6c736f61702e6f72672f736f61702f656e76656c6f70652f223e3c736f6170656e763a426f64793e3c4e53313a66696e6442794f7574706173734e6f20786d6c6e733a4e53313d22687474703a2f2f6f7574706173732e6170692e736572766963652e6669732e76323031352e656d617261746563682e636f6d2f312e30223e3c6f7574706173734e6f3e3230313330303030353831373c2f6f7574706173734e6f3e3c2f4e53313a66696e6442794f7574706173734e6f3e3c2f736f6170656e763a426f64793e3c2f736f6170656e763a456e76656c6f70653e''. The HTTP Reply Header bitstream (if any) received from the server was: 'X'485454502f312e3120343034204e6f7420466f756e640d0a436f6e74656e742d547970653a20746578742f68746d6c0d0a5472616e736665722d456e636f64696e673a206368756e6b65640d0a436f6e6e656374696f6e3a20436c6f73650d0a446174653a2053756e2c203132204d617920323031332030383a33353a353920474d540d0a5365727665723a20576562537068657265204170706c69636174696f6e205365727665722f372e300d0a0d0a''. The HTTP Reply Message Body bitstream (if any) received from the server was: 'X'3c48313e5352564530323535453a20412057656247726f75702f5669727475616c20486f737420746f2068616e646c65202f766973696f6e636f72652f4f7574706173735365727669636520686173206e6f74206265656e20646566696e65642e3c2f48313e3c42523e3c48333e5352564530323535453a20412057656247726f75702f5669727475616c20486f737420746f2068616e646c65207777772e766973696f6e2e61653a3930383020686173206e6f74206265656e20646566696e65642e3c2f48333e3c42523e3c493e49424d20576562537068657265204170706c69636174696f6e205365727665723c2f493e''. Ensure that the HTTP data is valid.
See the following messages for information pertaining to this error.
2013-05-12 12:46:21.361308 36404 RecoverableException BIP3691S: A SOAP request encountered an HTTP error while making an HTTP request. The HTTP request was made to the destination ''http://www.vision.ae:9080/visioncore/OutpassService''. The HTTP status line that was returned was: ''HTTP/1.1 404 Not Found''.
An error occurred during an HTTP request.
Check that the remote server is responding and that the request is specified correctly. See subsequent messages for information pertinent to this error.
2013-05-12 12:46:21.361340 36404 UserTrace BIP11507W: Rolled back a local transaction.
A local transaction has been rolled back for work done on the message flow thread.
2013-05-12 12:46:22.360978 36404 UserTrace BIP11501I: Received data from input node ''Outpass_EventIn''.
The input node ''Outpass_EventIn'' has received data and has propagated it to the message flow ''com.emaratech.vision.esb.event.outpass.VM_Outpass_Event''.
2013-05-12 12:46:22.361328 36404 UserTrace BIP2631I: Backed out message being propagated to failure terminal; node 'com.emaratech.vision.esb.event.outpass.VM_Outpass_Event.Outpass_EventIn'.
Node 'com.emaratech.vision.esb.event.outpass.VM_Outpass_Event.Outpass_EventIn' has received a message which has previously been backed out because of a processing error in the message flow. The MQMD 'backoutCount' of the message exceeds (or equals) the 'backoutThreshold' defined for the WebSphere MQ input queue. The message broker is propagating the message to the failure terminal of the node.
Examine the other messages and the message flow to determine why the message is being backed out. Correct this situation if possible. Perform any local error recovery processing required.
Regards,
Sarath |
|
| Back to top |
|
|
|
|
|
  |
Goto page 1, 2, 3 Next |
Page 1 of 3 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
