ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » IBM MQ API Support » WMQ 7 Client - Assured Delivery?

Post new topic  Reply to topic Goto page Previous  1, 2, 3, 4  Next
 WMQ 7 Client - Assured Delivery? « View previous topic :: View next topic » 
Author Message
bruce2359
PostPosted: Thu Mar 24, 2011 2:12 pm    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

Quote:
This is a great question.. I had that doubt too..

Can some one please clarify ?

Clarify what exactly?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Mar 24, 2011 3:07 pm    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

If the question is: is a 2009 r/c possible with server bindings?

However remote the possibility, I suppose it is. A connection is a connection is a connection.

The client kind depend much on the client platform- and network- stability. Server bindings are cross-memory, and not subject to the whims of the network.

I imagine that a server binding app could be torched by a storage violation or other deliberate tampering with mq internals that would result in a 2009.

I'll step back, and wait for someone from IBM to meander by with some more thoughts on this.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
mvic
PostPosted: Fri Mar 25, 2011 4:26 pm    Post subject: Reply with quote

Jedi

Joined: 09 Mar 2004
Posts: 2080

You can get a 2009 with a server bound app, although this is extremely rare, probably never seen, in practice. But if it happens, it's probably due to some fault within MQ or some odd OS behaviour (for example, someone running ipcrm).

People who CARE about recovering from this will develop some tolerance of failure / duplication into their apps. In the phrase we all know, avoid single points of failure! In this case avoid trusting everything to one piece of software.

Interested in your opinions on this:

Your app called MQCMIT, and because you're really unlucky, it returned 2009. OK, that gives you a problem, because you don't know if the failure was before or after the commit was enacted by the queue manager. You don't know if your work was done or not.

But, what about this. Your app calls MQCMIT which returns OK. Great! But then in the next line you have a coding error in your app, and it abends immediately. The MQ call worked, but your app abended. That's a problem equivalent to the MQCMIT returning with 2009. How do you handle it?

If you can handle the second scenario reliably it will be via some sort of independent reconciliation processing. ie. the app writes somewhere what it was about to do before it did it, and enough info to be able to find from elsewhere what happened to the piece of work. The same type of processing is what you need to recover from the 2009 from MQCMIT.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Sat Mar 26, 2011 5:57 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

WMQ is a pipe through messages flow. It is not WMQs problem if apps duplicate or lose messages; OR are not sensitive to the possibility of duplication or loss.

Application design best-practice (with WMQ or not) would be for the requesting application to keep track of the requests it sent, and the replies it received back from the replying app.

Likewise, the replying app would keep track of the requests it received, and the replies it sent back to the requesting program. By 'keep track' I mean in a db of some kind.

Both requesting and replying apps would thus be sensitive to the loss or duplication of messages. If the requesting app doesn't receive a reply, it should re-send. If the replying app receives duplicate requests, it would discard the dup (and notify the requesting app that a dup was received).

I'm not concerned with WMQ duplicating or losing messages; but I am concerned with apps doing so (losing data). This best-practice design model applies equally to WMQ- and non-WMQ applications.

Does this sound familiar? Am I repeating myself again?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Sat Mar 26, 2011 1:53 pm    Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

bruce2359 wrote:
...Application design best-practice (with WMQ or not) would be for the requesting application to keep track of the requests it sent, and the replies it received back from the replying app.


Yes, for the case of transactional request - reply messages, where DB updates (or maybe even interfaces to other downstream apps) need to be coordinated by the apps in multiple units of work.

This is not necessary for "inquiry" request - reply messaging, where no updates are made. If it fails, the app can just try again.
_________________
Glenn
Back to top
View user's profile Send private message
bruce2359
PostPosted: Sat Mar 26, 2011 2:00 pm    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

gbaddeley wrote:

Yes, for the case of transactional request - reply messages, where DB updates (or maybe even interfaces to other downstream apps) need to be coordinated by the apps in multiple units of work.

This is not necessary for "inquiry" request - reply messaging, where no updates are made. If it fails, the app can just try again.

I agree...unless the business requirement is for inquiries to be tracked (logged) for audit or other purposes.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
sunny_30
PostPosted: Tue Mar 29, 2011 10:11 am    Post subject: Reply with quote

Master

Joined: 03 Oct 2005
Posts: 258

bruce2359 wrote:
I imagine that a server binding app could be torched by a storage violation or other deliberate tampering with mq internals that would result in a 2009.

If the above statement is true, does that mean MQserver binding connection doesnt guarantee once-only assured delivery of a persistent message ?

My question is strictly related to MQ working here.
Things like ext application that does a MQget or MQput if it uses a back end database/ has audit setup/ how it recovers from 2009 error/ how it handles req-resp to track missing messages is a totally different topic
Back to top
View user's profile Send private message
bruce2359
PostPosted: Tue Mar 29, 2011 10:34 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

sunny_30 wrote:
bruce2359 wrote:
I imagine that a server binding app could be torched by a storage violation or other deliberate tampering with mq internals that would result in a 2009.

If the above statement is true, does that mean MQserver binding connection doesnt guarantee once-only assured delivery of a persistent message ?

My hypothetical scenario involved a wicked sysadmin hacking the running wmq image, leading to a theoretical 2009.
Assured delivery is ensured by a running qmgr in the usual, well-documented ways. What happens, do you suppose, when a unit of work is neither explicitly committed nor explicitly backed out by the application?

What happens, do you suppose, when the qmgr restarts, and discovers a unit of work that did not complete?

sunny_30 wrote:
My question is strictly related to MQ working here.
Things like ext application that does a MQget or MQput if it uses a back end database/ has audit setup/ how it recovers from 2009 error/ how it handles req-resp to track missing messages is a totally different topic


If you question is 'how does wmq recover from ReasonCode 2009?' WMQ doesn't recover from any CompletionCode or ReasonCode. The CC and RC are presented to the application. If the application fails, then WMQ will take the well-documented actions ensure that queues and other resources are restored to their last known consistent state.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
sunny_30
PostPosted: Tue Mar 29, 2011 11:03 am    Post subject: Reply with quote

Master

Joined: 03 Oct 2005
Posts: 258

PeterPotkay wrote:
When you get a 2009 on an MQCMIT call, how do you know whether it worked or not in committing the message?

My question is also the same but imagine here the App is doing MQCMIT during PUT using local server-binding connection (a two phase commit?)

In this scenario, as the app never received the response due to the 'theoretical 2009', obviously the app would try the same PUT again resulting in data duplication

Howz this hypothetical scenario, even though very rare, prevented at MQ level ? Does MQ guarantee once-only assured delivery ?

Im not interested to know how QMgr recovers a persistent message sitting on (a damaged) queue after its restart replayed from a Linear log etc etc... & also Im not intersted in how the PUT app recovers from 2009 RC.. (however i imagine it PUTs the message again .. )
Back to top
View user's profile Send private message
mvic
PostPosted: Tue Mar 29, 2011 11:35 am    Post subject: Reply with quote

Jedi

Joined: 09 Mar 2004
Posts: 2080

sunny_30 wrote:
In this scenario, as the app never received the response due to the 'theoretical 2009', obviously the app would try the same PUT again resulting in data duplication

First of all, this theoretical 2009 from commit is a general problem with single-phase commit APIs, and is common to any product that offers a single-phase commit.

Secondly, you have precisely the same problem in principle if you have a coding error in your app in the line after the commit call returns.

Quote:
Howz this hypothetical scenario, even though very rare, prevented at MQ level ? Does MQ guarantee once-only assured delivery ?

Assuming I understood your question, the answer is there is no absolute guarantee with single-phase commit. This is inherent to what single-phase commit is. And the principle is general, not specific to MQ.

But it is possible I misunderstood what you were asking. (?)
Back to top
View user's profile Send private message
bruce2359
PostPosted: Tue Mar 29, 2011 11:50 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

Quote:
In this scenario, as the app never received the response due to the 'theoretical 2009', obviously the app would try the same PUT again resulting in data duplication

Assured message delivery means that WMQ will deliver a message once and once only, with no loss or duplication. WMQ is only a transport mechanism. Applications are free to put zero, one, or more duplicate messages; and free to lose messages.

This is where a well-written application (with or without WMQ as the transport) comes into play.

A well-written requesting app will keep track of the requests it has sent, and the replies it received from the responder. The responding app will keep track of the requests it received, and the replies it returned to the requester.

Both requesting and responding well-written apps must be sensitive to loss or duplication of data (messages in mq-speak). The requesting end must be written to re-send requests if replies were never returned; and to reject duplicate replies. The responding end must be written to reject duplicate requests; and to notify the requester if it received duplicate requests.

Your question: A bindings app suddenly and mysteriously receives a 2009 after MQCMIT call (a two-phase commit), BUT before the app receives the CC/RC. Was the commit successful? Or not?

More specifically, what will WMQ do if the MQCMIT call was successful, but the CC/RC could not be delivered successfully to the app?

Hmmmm. I'll wait a bit if someone at IBM cares to join us in this hypothetical.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
mvic
PostPosted: Tue Mar 29, 2011 12:12 pm    Post subject: Reply with quote

Jedi

Joined: 09 Mar 2004
Posts: 2080

bruce2359 wrote:
More specifically, what will WMQ do if the MQCMIT call was successful, but the CC/RC could not be delivered successfully to the app?

This is exactly the case we have been talking about. And it is an inherent problem with any single-phase commit API. Unless someone can correct me...
Back to top
View user's profile Send private message
mqjeff
PostPosted: Tue Mar 29, 2011 12:18 pm    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

I was quite sure that MQ came with plenty of Magic Server Beans to solve just such problems.

And it would collapse NP into P with the proper configuration, too.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Tue Mar 29, 2011 2:42 pm    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

I don't think it makes any difference whether its single phase or 2 phase commit. An app does an MQPUT. Whether it gets a 2009 on its MQCMIT in the case of a single phase, or the transaction manager gets it in a 2 phase, your app or the transaction manager has the same dilemma - did the message get committed to the queue or not, right?
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
mvic
PostPosted: Tue Mar 29, 2011 3:10 pm    Post subject: Reply with quote

Jedi

Joined: 09 Mar 2004
Posts: 2080

PeterPotkay wrote:
Whether it gets a 2009 on its MQCMIT in the case of a single phase, or the transaction manager gets it in a 2 phase, your app or the transaction manager has the same dilemma - did the message get committed to the queue or not, right?

Well, not really, IMHO.

In 2PC there are post-crash recovery actions that are taken, so the app has an even more solid safeguard than 1PC (which is already solid enough for most people, of course).

In a 2PC post-crash scenario, the TM asks the RMs, "what transactions do you think are still pending". The RMs tell the TM, and the TM then informs the RMs of the proper outcome, which it has of course remembered in its logs.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page Previous  1, 2, 3, 4  Next Page 2 of 4

MQSeries.net Forum Index » IBM MQ API Support » WMQ 7 Client - Assured Delivery?
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.