| Author |
Message
|
| GeneRK4 |
 Posted: Sat Feb 22, 2014 8:54 pm Post subject: Message loss in HTTP call |
 |
|
Master
Joined: 08 Jul 2013
Posts: 220
|
I have a design like below,
HTTPInput node --> Database node --> MQOutput node
During failure scenarios,the failure messages are placed in Failure queues.
Hence there are no message loss during failure scenarios.
During Success scenarios,we recently found message loss in Production.
This situation happened when there was some process shutdown happened due to abend files creation.
I guess if I introduce MQ (persistant queue) in the middle of HTTPInput node and Database node,then even during abend files creation and process interrupting unexpectedly,the message will not be lost.
Please correct this if this is wrong..
I also see,SYSTEM.HTTP.WS.INPUT queue which gets message once HTTPlistener receives the message.After this system queue,message passes through the flow.
When I try to stop the flow in the middle of message transit,I am not able to capture message in this system queue.Hence,I guess,if I dont introduce a queue explicitly in the middle of HTTP and Database,there is no way that I can capture the message during this interruption scenarios.
Please suggest your thoughts.. |
|
| Back to top |
|
 |
| Gralgrathor |
| Posted: Sun Feb 23, 2014 3:42 am Post subject: Re: Message loss in HTTP call |
|
|
Master
Joined: 23 Jul 2009
Posts: 297
|
| GeneRK4 wrote: |
| When I try to stop the flow in the middle of message transit |
What do you mean? Are you saying that you would like MQ to be able to compensate for *everything* that can go wrong between the initial request and its processing?
Your request enters the flow through HTTPInput, and is put to an explicitly defined queue. If you put your HTTPReply to send a response only when you know the message has been stored, then the sending application will know when something's gone wrong and enter a retry or compensation handler.
You also have to allow for the possibility that systems fail, and messages are sometimes lost in transit. Your solution should take this into account: there must be end to end checks and compensations, not just in the MQ/WMB part of the chain.
_________________
A measure of wheat for a penny, and three measures of barley for a penny; and see thou hurt not the oil and the wine. |
|
| Back to top |
|
|
| GeneRK4 |
| Posted: Sun Feb 23, 2014 6:46 pm Post subject: |
|
|
Master
Joined: 08 Jul 2013
Posts: 220
|
Thanks..
I want to know how message persistance can be achieved in HTTP mode of protocol.
http://publib.boulder.ibm.com/infocenter/wmbhelp/v6r1m0/index.jsp?topic=%2Fcom.ibm.etools.mft.doc%2Fac00420_.htm helps me for MQ,MQTT.But not for other protocols.
I tried to achieve message persistance by modifying Persistance property to "yes" in SYSTEM.BROKER.WS.INPUT queue.I could see when I abruptly stop the Execution group,the message is logged in active logs of MQ.I guess this would help me in achieving message persistance in HTTP mode of protocol as well.But I guess we should have trade-off between Message persistance and Performance(as we persist each and every message hitting at HTTP Listener port).
Please correct me if I am wrong. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Feb 23, 2014 10:30 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20703
Location: LI,NY
|
| GeneRK4 wrote: |
Thanks..
I want to know how message persistance can be achieved in HTTP mode of protocol.
http://publib.boulder.ibm.com/infocenter/wmbhelp/v6r1m0/index.jsp?topic=%2Fcom.ibm.etools.mft.doc%2Fac00420_.htm helps me for MQ,MQTT.But not for other protocols.
I tried to achieve message persistance by modifying Persistance property to "yes" in SYSTEM.BROKER.WS.INPUT queue.I could see when I abruptly stop the Execution group,the message is logged in active logs of MQ.I guess this would help me in achieving message persistance in HTTP mode of protocol as well.But I guess we should have trade-off between Message persistance and Performance(as we persist each and every message hitting at HTTP Listener port).
Please correct me if I am wrong. |
Forget about even attempting it. HTTP or HTTPS are inherently unreliable protocols. What happens if the message does not even reach your endpoint? (routing, firewall, etc...)?
Also in a fire and forget scenario you would only be able to send the ok header if you managed to successfully deal with the message....
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| GeneRK4 |
| Posted: Sun Feb 23, 2014 11:02 pm Post subject: |
|
|
Master
Joined: 08 Jul 2013
Posts: 220
|
Ok..Thank you very much 
The customer is not getting convinced with WMB flow for such critical transactions.Thatsy was looking for such options. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Feb 24, 2014 12:43 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
If the customer doesn't want to lose critical transactions, the customer shouldn't be using HTTP.
HTTP is an unreliable protocol that is designed to allow for the loss of messages. |
|
| Back to top |
|
|
| GeneRK4 |
| Posted: Mon Feb 24, 2014 1:00 am Post subject: |
|
|
Master
Joined: 08 Jul 2013
Posts: 220
|
|
| Back to top |
|
|
| Gralgrathor |
| Posted: Mon Feb 24, 2014 11:08 am Post subject: |
|
|
Master
Joined: 23 Jul 2009
Posts: 297
|
| GeneRK4 wrote: |
| The customer is not getting convinced with WMB flow for such critical transactions |
The customer thinks HTTP is more reliable than MQ?
You gotta get some IBM expert to explain some things to the customer, I think. Perhaps somebody once told them that WS-ReliableMessaging is the next hot thing and now they've got this screwy idea of a reliable SOA messaging system...
_________________
A measure of wheat for a penny, and three measures of barley for a penny; and see thou hurt not the oil and the wine. |
|
| Back to top |
|
|
| mqsiuser |
| Posted: Tue Feb 25, 2014 1:02 am Post subject: |
|
|
Yatiri
Joined: 15 Apr 2008
Posts: 637
Location: Germany
|
| fjb_saper wrote: |
| Forget about even attempting it. HTTP or HTTPS are inherently unreliable protocols. What happens if the message does not even reach your endpoint? (routing, firewall, etc...)? |
HTTP and HTTPS is not so bad 
OFC if this is your incoming endpoing (and you don't control the sender side (with broker, or whatever)), then the sending side may just not deal with the (unreliable) nature of HTTP(S) (properly).
But always remember: You can make up certain insufficencies in an underlying (network protocol) layer (on the next (upper) layer)... including this one (reliabilitly)
E.g. When you use Message Broker it can handle the 'HTTP(S) timeout' and 'HTTP return codes' and make the HTTP(S) call/connection reliable.
Pls correct me if I am wrong.
_________________
Just use REFERENCEs |
|
| Back to top |
|
|
|
|
