ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexIBM MQ API SupportWMQ 7 Client - Assured Delivery?

Post new topicReply to topic Goto page 1, 2, 3, 4  Next
WMQ 7 Client - Assured Delivery? View previous topic :: View next topic
Author Message
hopsala
PostPosted: Thu Aug 05, 2010 9:41 am Post subject: WMQ 7 Client - Assured Delivery? Reply with quote

Guardian

Joined: 24 Sep 2004
Posts: 953

So we all know that MQ Client isn't assured delivery... Or do we?

http://www.mqseries.net/phpBB2/viewtopic.php?p=164433
http://www.mqseries.net/phpBB2/viewtopic.php?t=26623
http://fixunix.com/websphere/211620-guarantee-delivery-mq-client.html

These are all old posts. I am unable to find any new posts about this, nor any reference to this topic in the MQ7 info center. Is MQ client now assured delivery? No more 2009 duplicate message probs? If so, since when? If not, has IBM ever issued some sort of a statistics or formal provisioning explaining how to minimize the risk?
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Aug 05, 2010 9:53 am Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 7830
Location: US: west coast, almost. Otherwise, enroute.

"Assured delivery" means that a message will be delivered once only - with no loss or duplication.

This applies equally to client-bound and server-bound apps. The client is designed for platforms where a server is not required/needed/available.

Being a non-qmgr implementation, the client does not have queues and logging - these are on the server.

What this means is that client-bound applications are not as robust due to things like the network that separates the client and qmgr, and the apps ability/inability to redrive a lost connection (better in v7).
_________________
I didn't know that Schrdinger had a cat.
Back to top
View user's profile Send private message
hopsala
PostPosted: Thu Aug 05, 2010 10:27 am Post subject: Reply with quote

Guardian

Joined: 24 Sep 2004
Posts: 953

Hey Bruce. No offense, but I'm not quite sure why you just explained all that you did, as I did not ask about any of it. If I did not make myself clear, I apologize: When I said "client is not assured delivery" I simply meant "when using mq client messages are not assured to be delivered once and only once"

The links I posted quite explicitly state that MQ client is not assured delivery - you can definitely have a message sent twice, or not at all, without your code knowing about it, albeit in a very specific and admittedly rare scenario. I also know this from experience, from conversations with IBMers, and from lecturing at many MQ administration courses, but last time I heard anything about it was over three years ago. All I'm asking now is - Has anything changed in V7? You say it's "better" in V7, better how? Does it still happen?

Thanks
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Aug 05, 2010 11:18 am Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 7830
Location: US: west coast, almost. Otherwise, enroute.

Quote:
Hey Bruce. No offense

None taken.

Quote:
...but I'm not quite sure why you just explained all that you did, as I did not ask about any of it.

I don't know exactly what you do know; so I offer some basics and some background. I offered IBMs official definition of "assured delivery" as it is defined in the official IBM APR and APG. Assured delivery is provided by the qmgr, and not by the client. This has not changed in the v7 client.

Quote:
The links I posted quite explicitly state that MQ client is not assured delivery - you can definitely have a message sent twice, or not at all, without your code knowing about it, albeit in a very specific and admittedly rare scenario.

Yep, rare scenarios. This is why the client is a choice; and like all choices, some thought, planning and design are required.
I'll repeat: the client does not, and has never assured delivery. Assured delivery is done by qmgrs.

Briefly, and for clarity:
1) a client app issues an API call
2) the WMQ client layer ships the API call (and message if the call is MQPUT) to the SVRCONN channel
3) the SVRCONN channel agent receives the API call, and
4) acts as a proxy for the client, and
5) executes the API call (as if it was issued by a server-bound app
6) at call completion, the agent returns to the client app ReasonCode, CompletionCode (and message if MQGET call)

Nothing in this scenario suggests loss or duplication of the message.

Where the client connection offers risk, for example:
After the client app issues the call, and the agent executes it, the client app abends or the network connection fails BEFORE the agent can return the RC/CC/message.

I will agree that if the client app abends (or the platform o/s fails), it cannot know the outcome of the MQI call.

If the call was MQPUT, the message is in a queue. If the call was MQGET and the message was not in a UofW, it is possible that the message is lost.

It is also possible that the client app lives on, but the qmgr fails. In this case, the app receives an appropriate RC/CC.

A best-practice for request-reply scenarios has been discussed here many times. It obligates the requesting app to keep track of what request messages it sent, and replies it received; and to recognize and deal with duplicated (not by mq) or missing replies. Likewise, the replying app is obligated to recognize duplicate or missing (again, not necessarily by mq) requests; and deal with them.

Your written tone (pardon me if I've misread this) demonstrates your surprise and annoyance with the WMQ Clients' well-known and well-documented capabilities and limitations.

[edit]
Quote:
All I'm asking now is - Has anything changed in V7? You say it's "better" in V7, better how? Does it still happen?

This also is well-documented. Do some research.
_________________
I didn't know that Schrdinger had a cat.
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Thu Aug 05, 2010 3:55 pm Post subject: Reply with quote

Padawan

Joined: 25 Mar 2003
Posts: 1719
Location: Melbourne, Australia

Quote:
I will agree that if the client app abends (or the platform o/s fails), it cannot know the outcome of the MQI call.


This is the crux of why MQ Client does not provide assured delivery. To provide this level of service the MQ Client protocol would need to include MCA level transactionality, like distributed or cluster channels do, and be able to report to the app program the ultimate success or failure of an MQI call.
_________________
Glenn
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Aug 05, 2010 4:19 pm Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 7830
Location: US: west coast, almost. Otherwise, enroute.

Quote:
This is the crux of why MQ Client does not provide assured delivery.

I am painfully trying to be more explicit as to the concept of "assured delivery," and as to where "assured delivery" support exists - which is only the qmgr - where logging, UofW, restart, and a fleet of other internal code assures once and once-only delivery.

The OP appears to be once-again target-locked, this time on assured delivery.

The client layer, lacking logging, et.al., has not, does not, and cannot not implement assured delivery. A client app can, to a limited degree, and with good app design and coding, minimize the potential for client app problems.

So-called "lost" messages has been well-documented here and in the manuals.

Some of the more interesting sources of lost messages include:
queue-full condition that the app failed to catch
messages with expiry values
messages that end up in the dlq (for the well-documented reasons)
non-persistent messages transmitted across NPMSPEED(FAST) channels, where the destination queue is full, put-inhibited, too big for the queue, and the dlq doesn't exist, is full, put-inhibited, etc..

[edit]
My experience is that MQ does not lose messages. Rather, messages are lost due to:
- failure to read and understand the APR, APG, SAG, and other relevant doc
- poor application design and coding
- failure to understand the behavior of API calls (manuals)
- failure of the app to catch errors (manuals, pardon the redundancy)
- failure to adhere to best-practice
- failure to adequately test
[/edit]

Others have bitterly complained that lost messages demonstrate that "mq sucks," and "assured delivery is a hoax." (I quote a former client.)

Does anyone remember sami.stormrage?
_________________
I didn't know that Schrdinger had a cat.
Back to top
View user's profile Send private message
mvic
PostPosted: Thu Aug 05, 2010 4:47 pm Post subject: Re: WMQ 7 Client - Assured Delivery? Reply with quote

Padawan

Joined: 09 Mar 2004
Posts: 1981

hopsala wrote:
Is MQ client now assured delivery?

I assume this is a genuine question, not a debating point. If so, please write a lot more about what you mean.

Quote:
No more 2009 duplicate message probs?

Huh? Of course you can get 2009. And if you get a duplicate message via a client, it was because the application chose to re-generate the same message. In some scenarios a duplicate message can be detected and handled, so is not a disaster. In other scenarios that can't handle it, it is a disasater - so the client should be tentative and not generate duplicates. Again, what did you mean by this simple one-clause sentence?

Quote:
... has IBM ever issued some sort of a statistics or formal provisioning explaining how to minimize the risk?

Not that I know of. But I am not sure of your requirements, or if this is a debating point.

MQ is well-designed software. But don't expect it to perform miracles.
Back to top
View user's profile Send private message
mvic
PostPosted: Thu Aug 05, 2010 4:53 pm Post subject: Reply with quote

Padawan

Joined: 09 Mar 2004
Posts: 1981

gbaddeley wrote:
This is the crux of why MQ Client does not provide assured delivery.

Assured delivery is note a useful phrase when you are talking technical details.

If your connection fails during MQCMIT processing you have some doubt about the outcome.

There is a similar (but smaller) window of uncertainty in a server-bound application too.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Aug 05, 2010 5:54 pm Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 7830
Location: US: west coast, almost. Otherwise, enroute.

Quote:
No more 2009 duplicate message probs?

This question makes no sense. 2009 is a connection broken ReasonCode. The presence or absence of a 2009 in no way indicates a duplicate or lost message, only that the connection is broken.

It is the responsibility of the application to catch all relevant/possible ReasonCodes, and deal with them.
_________________
I didn't know that Schrdinger had a cat.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Sat Aug 07, 2010 1:11 pm Post subject: Reply with quote

Jedi Council

Joined: 15 May 2001
Posts: 7372

bruce2359 wrote:
Quote:
No more 2009 duplicate message probs?

This question makes no sense. 2009 is a connection broken ReasonCode. The presence or absence of a 2009 in no way indicates a duplicate or lost message, only that the connection is broken.


Makes sense to me.

If you get a MQRC 2009 on your MQCMIT after doing an MQPUT, did the message get committed? If you guess no and you guess wrong and you do another put of the same message......viola! duplicate message problem.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
bruce2359
PostPosted: Sat Aug 07, 2010 2:18 pm Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 7830
Location: US: west coast, almost. Otherwise, enroute.

Quote:
Makes sense to me.

If you get a MQRC 2009 on your MQCMIT after doing an MQPUT, did the message get committed? If you guess no and you guess wrong and you do another put of the same message......viola! duplicate message problem.


I agree completely.

Does 2009 mean message has been lost or duplicated? No.

It does mean that the client app must be sensitive to the lost connection, AND what that might mean to a message.

If you guess?! If you guess? (read my astonishment here).

If you guess that 2033 means the queue is empty, bad guess. If you ignore a 2051 or 2035, and guess (presume) that a message was successfully put, you guessed wrong here, too.

The OP wants assured (guaranteed?) delivery from the mq client, and apparently doesn't want to any app coding to improve reliability of the app.

[edit]
WMQ Client is a choice (do I repeat myself?). The decision must be based on risk/benefit analysis. If it is not possible to have a qmgr at the application platform, then it is incumbent on the app design to anticipate things that can misbehave, AND develop methodologies to deal with them.

Guesswork is not part of a good business application design.
_________________
I didn't know that Schrdinger had a cat.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Sat Aug 07, 2010 11:03 pm Post subject: Reply with quote

Jedi Council

Joined: 15 May 2001
Posts: 7372

When you get a 2009 on an MQCMIT call, how do you know whether it worked or not in committing the message?

Obviously I didn't mean your app should have a little heads or tails simulator coded to take random action. Perhaps a better choice of words would be, if you get a 2009 on an MQCMIT call, you must decide in advanced what you will assume that means, and your app is coded to do that.

That could mean you assume the message made it, and you code to not resend it (guess, I mean, decide and code wrong and missing message problem).

That could mean you assume the message never made it, and you code to resend it (decide wrong and duplicate message problem).

Or you code yourself to halt any and all further processing related to that specific message and raise an alert that kicks off manual intervention. Since you never got a good RC from MQ on that MQCMIT, the app must still assume responsibility over that message data and so should have that data to save off somewhere in the meantime while problem resolution commences. Perhaps a lot of work, but depending on the nature of the data involved, perhaps critical. And perhaps maybe not worth the time and effort compared to just putting a local QM there instead and letting the internal MQ code deal with it.


I would love to see worldwide what all those MQ Client apps out there are coded to do when 2009 shows up on an MQCMIT. Sadly a good portion of them probably don't even check the MQRC.

On a related note, I don't think 2009 is a RC specific to MQ Clients. Is it not possible, although far, far more less likely, for it to occur for an app connected to the QM in bindings mode? Can't that connection somehow break and throw a 2009?
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
zpat
PostPosted: Sun Aug 08, 2010 12:07 am Post subject: Reply with quote

Jedi Council

Joined: 19 May 2001
Posts: 5515
Location: UK

How does the ETC affect this? (Extended Transactional Client)

This comes free with WAS 6 as it happens.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Sun Aug 08, 2010 6:02 am Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 7830
Location: US: west coast, almost. Otherwise, enroute.

Quote:
Sadly a good portion of them probably don't even check the MQRC.

One developer told me that there were just too many RCs to check.

Quote:
How does the ETC affect this?

I'm hoping to be corrected here, but I don't believe the ETC offers anything more than, as advertised, XA-compliancy.
_________________
I didn't know that Schrdinger had a cat.
Back to top
View user's profile Send private message
sunny_30
PostPosted: Thu Mar 24, 2011 1:10 pm Post subject: Reply with quote

Master

Joined: 03 Oct 2005
Posts: 252

I had some questions reg data delivery assurance using MQv7client
Lot of good info here

However, above, PeterPotkay asked->
Quote:
Is it not possible, although far, far more less likely, for it to occur for an app connected to the QM in bindings mode? Can't that connection somehow break and throw a 2009?

This is a great question.. I had that doubt too..

Can some one please clarify ?

If the answer is YES, then how does a binding connection ensure no data loss on MQget & no data duplication on MQput ?
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Goto page 1, 2, 3, 4  Next Page 1 of 4

MQSeries.net Forum IndexIBM MQ API SupportWMQ 7 Client - Assured Delivery?
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.