| Author |
Message
|
| atheek |
 Posted: Tue Apr 01, 2008 5:11 pm Post subject: MDB picking messages out of sequence |
 |
|
Partisan
Joined: 01 Jun 2006
Posts: 327
Location: Sydney
|
Hi All
We have an application in production that uses a MDB to receive inbound messages from a MQ. Now, recently the requirement changed that all messages that the remote system puts to that queue should be in sequence. The message payload has a sequence ID field and we are checking this in our MDB and if there is an error, the message will be rolled back to the queue and an alert will be raised.
Today, we got an issue. The error we got was something like
Expected Sequence No : "5162786, but received 5162793" . The message was rolled back and MDB was paused.
I browsed the queue and found that messages were in this order
5162787
5162788
5162789
5162790
5162791
5162792
5162793
out of which 5162787-5162792 has a backout count of 0, while 5162793 has a backout count of 1. No clue what happened to 5162786. Also no idea why mdb picked 5162793 ahead of other messages?
Is this any known bug or am I missing anything?
MQ Version 6.0.2.0
MDB deployed in weblogic server 9.2
TIA
-Atheek |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Tue Apr 01, 2008 5:33 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
This is why you should not have message affinity.
If you need to process in sequence
a) make your messages persistent to make sure none is lost to a poor connection
b) put them into a DB and process them delayed so that you can retrieve them from the DB and process them in order...
Now if your sequence is not absolute but relative to a group you might want to look at message grouping...
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| atheek |
| Posted: Tue Apr 01, 2008 6:21 pm Post subject: |
|
|
Partisan
Joined: 01 Jun 2006
Posts: 327
Location: Sydney
|
thanks fjb_saper. The messages are persistent and we dont have a choice to use database 
I dont have much knowledge into how MDB listeners work...like to know whether any one have faced such a issue before and resolved it |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Apr 01, 2008 6:24 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| atheek wrote: |
thanks fjb_saper. The messages are persistent and we dont have a choice to use database
I dont have much knowledge into how MDB listeners work...like to know whether any one have faced such a issue before and resolved it |
Then you need to put your foot or  down and challenge the sanity of the design. Why ? Because you have just painted yourself into a corner with a design that will not scale and limit your throughput
Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Apr 02, 2008 12:23 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| atheek wrote: |
| I dont have much knowledge into how MDB listeners work...like to know whether any one have faced such a issue before and resolved it |
It's not a problem with MDB listeners, it's a problem with your design which would manifest sooner or later with any message-reading technology you use, not just MDB.
You have a design that has message affinity. I would query the design being insane, but it's certainly crackers and I agree that it doesn't scale at all.
If you're determined to process these messages in sequence, you have 2 options. The sensible one is to use a database, which you say is not an option for you. The other is to deliberately read them off the queue in sequence, rather than reading them in the order they're presented by the queue manager. So rather than a straight MDB, code something that browses the queue for the next sequence number, does a get under cursor and then repeats.
If you have any volume of throughput at all, this method will suck very quickly. Very quickly indeed. Even if you haven't, it's going to be rather resource heavy.
In your place, I'd adopt the strategy suggested by [b]fjb_saper[/] and trout whoever came up with this design. I've changed by mind - it is insane!
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mvic |
| Posted: Wed Apr 02, 2008 3:22 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| fjb_saper wrote: |
a) make your messages persistent to make sure none is lost to a poor connection
b) put them into a DB and process them delayed so that you can retrieve them from the DB and process them in order... |
On point "a" I agree. On point "b" I don't agree this is necessary. MQ guarantees message ordering, as per http://publib.boulder.ibm.com/infocenter/wmqv6/v6r0/topic/com.ibm.mq.csqzal.doc/fg12610_.htm
Note that in order to preserve ordering the messages must all be retrieved by MQGET calls from the same HConn. This implies that only one HConn is getting from the queue on which ordering requirements are being expected. This may or may not be compatible with the getting application being an MDB; don't know.
A couple of defects were found in MQ's support for ordering, but these are fixed in 6.0.2.3. Ref APAR IZ00896.
There may, of course, be wider benefits from going via a database as was described above, but getting messages in the same order they were put need not be one of them. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Apr 02, 2008 3:32 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
One of the conditions that link specifies is:
| Quote: |
| the queue is local to the putting application |
As soon as the messages go over a channel, all bets are off.
There's also no guarantee in this specific example that all the messages are outside a unit of work, or in the same one. In any event, designing an application system which meets all these requirements is foohardy as it won't scale & is vunerable to someone coming along in 6 months time & changing one of these criteria.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mvic |
| Posted: Wed Apr 02, 2008 3:40 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| Vitor wrote: |
| As soon as the messages go over a channel, all bets are off. |
It's still possible to preserve ordering I believe, but things do become more difficult when a channel is involved, yes. I've heard that some do this with channels, but to preserve ordering I think you have to do without Dead Letter Queues on all the queue managers from putter to getter. That's not a nice situation to be in if things go a bit wrong and the DLQ is needed. |
|
| Back to top |
|
|
| mvic |
| Posted: Wed Apr 02, 2008 3:43 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| Vitor wrote: |
| There's also no guarantee in this specific example that all the messages are outside a unit of work or in the same one. |
Applications can be coded accordingly, to give the guarantee. Did I misunderstand your point?
| Quote: |
| In any event, designing an application system which meets all these requirements is foohardy as it won't scale & is vunerable to someone coming along in 6 months time & changing one of these criteria. |
That's up to the business and engineering controls. I agree, if these are not adequate to prevent such occurrences, then the occurrences might well occur.  |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Apr 02, 2008 3:50 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mvic wrote: |
| Vitor wrote: |
| As soon as the messages go over a channel, all bets are off. |
It's still possible to preserve ordering I believe, but things do become more difficult when a channel is involved, yes. I've heard that some do this with channels, but to preserve ordering I think you have to do without Dead Letter Queues on all the queue managers from putter to getter. That's not a nice situation to be in if things go a bit wrong and the DLQ is needed. |
What you're doing is single threading your entire infrastructure so the first time anything goes wrong that might impact message ordering, the whole thing stops and waits for you to fix it. The implication is that you have the sequenced messages using their own queue managers etc so as not to impact SLA on the rest of the message domain.
There are very few scenarios where the affinity is that rigid that such a set up is cost-effective, or more cost effective than a quick re-write and a database license.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Apr 02, 2008 3:53 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mvic wrote: |
| Vitor wrote: |
| There's also no guarantee in this specific example that all the messages are outside a unit of work or in the same one. |
Applications can be coded accordingly, to give the guarantee. Did I misunderstand your point? |
My point was that we don't know how this application is coded. It could be putting in multiple units of work for all we know, bound to a database at the putting end perhaps. There no information in the post was simply my point.
| mvic wrote: |
| Vitor wrote: |
| In any event, designing an application system which meets all these requirements is foohardy as it won't scale & is vunerable to someone coming along in 6 months time & changing one of these criteria. |
That's up to the business and engineering controls. I agree, if these are not adequate to prevent such occurrences, then the occurrences might well occur. |
Controls? Design standards? Impact analysis? Oh speak to me of such wonders! 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| atheek |
| Posted: Wed Apr 02, 2008 4:09 am Post subject: |
|
|
Partisan
Joined: 01 Jun 2006
Posts: 327
Location: Sydney
|
Thanks to all ! I believe the problem has to do with the MQ bug as pointed by mvic ...(IZ00896). We may go for an upgrade of MQ.
I know abt this message affinity issue before from this forum, but we dont have much choice..thats how our application is architectured and its wasn't my call
-Atheek |
|
| Back to top |
|
|
|
|
