| Author |
Message
|
| cse.gokul |
 Posted: Wed Jul 22, 2015 7:14 am Post subject: MQ Message coming out of sequence |
 |
|
Newbie
Joined: 22 Jul 2015
Posts: 5
|
My J2EE application has 2 QCFs for load balancing and for failover. The J2EE app puts message 1, does some processing and puts message 2 to queues via QCF1/QCF2. Both QCFs connects to the its queue managers and alias queues. And then queues messages are routed to a common single local queue which another app consumes.
In some cases, message 2 is coming before message 1 in the common local queue. If I check the messages in the queue, I can see the timestamp and identify that message 2 has sequence number before message 1.
Consumer is dependent on the sequence of the message and it is failing due to this.
Could you suggest what can be done at MQ end or J2EE app to maintain the sequence? |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Wed Jul 22, 2015 7:21 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Have you tried putting both messages in a group? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| cse.gokul |
| Posted: Wed Jul 22, 2015 7:26 am Post subject: |
|
|
Newbie
Joined: 22 Jul 2015
Posts: 5
|
| I cannot group it as application can fail during the processing between the two put. So that atleast 1 put is done in case of any failures. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Jul 22, 2015 7:30 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| cse.gokul wrote: |
| I cannot group it as application can fail during the processing between the two put. So that atleast 1 put is done in case of any failures. |
Consider sending them in a unit of work. i.e. both messages or none.
[moving to Java forum]
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 22, 2015 7:45 am Post subject: Re: MQ Message coming out of sequence |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| cse.gokul wrote: |
| Consumer is dependent on the sequence of the message and it is failing due to this. |
And this is one reason why message affinity (the processing of a message being dependent on the message received before or after it) is a bad practice design.
Another is that it doesn't scale. The consumer can only ever have one reading thread as there's no way to assure that message 2 will be picked up by the same thread that read message 1.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 22, 2015 7:51 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| fjb_saper wrote: |
| Consider sending them in a unit of work. i.e. both messages or none. |
Assuming that message 2 isn't dependent on message 3 in the same way message 2 is on message 1. Also the OP seems to indicate that message 1 is required even if there's a failure producing message 2.
Another answer, which is perhaps more general, is to only use a single QCF. If your consumer can't load balance, why should the producer? A single threaded producer ensures message sequence.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Jul 22, 2015 7:57 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Vitor wrote: |
| A single threaded producer ensures message sequence. |
Assuming a single route through the queue manager network, and a single destination. |
|
| Back to top |
|
|
| cse.gokul |
| Posted: Wed Jul 22, 2015 7:59 am Post subject: |
|
|
Newbie
Joined: 22 Jul 2015
Posts: 5
|
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 22, 2015 8:56 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqjeff wrote: |
| Vitor wrote: |
| A single threaded producer ensures message sequence. |
Assuming a single route through the queue manager network, and a single destination. |
Assuming that.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Jul 22, 2015 11:33 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Vitor wrote: |
| mqjeff wrote: |
| Vitor wrote: |
| A single threaded producer ensures message sequence. |
Assuming a single route through the queue manager network, and a single destination. |
Assuming that. |
Assuming on top of that that the messages in the same group have all the same priority, or that all queues involved, including xmit queues are set to FIFO for msg delivery sequence... (default is priority)
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Wed Jul 22, 2015 5:45 pm Post subject: Re: MQ Message coming out of sequence |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| cse.gokul wrote: |
| Could you suggest what can be done at MQ end or J2EE app to maintain the sequence? |
MQ does not guarantee message sequence, except in very specific circumstances. Do not design apps to rely on this!
Sequencing should be built into the consuming app design to perform message order marshalling, eg. use a database to store messages until all messages in the required sequence have arrived. It also needs to cater for missing messages.
Another approach is to bundle all the sequenced data into a single message or a message "group".
_________________
Glenn |
|
| Back to top |
|
|
| cse.gokul |
| Posted: Thu Jul 23, 2015 7:36 am Post subject: |
|
|
Newbie
Joined: 22 Jul 2015
Posts: 5
|
| we finalized to change the consumer to remove dependency of the sequence. Thanks for the inputs again. |
|
| Back to top |
|
|
| apatnaik14 |
| Posted: Wed Aug 12, 2015 1:17 pm Post subject: |
|
|
Newbie
Joined: 11 Aug 2015
Posts: 8
|
I feel like the dependency was in place to identify and map the request and response pair or even just the request messages, since you made it clear above that request 2 is not dependent on the output of request 1.
In that case, you can just use the correlationId to identify the request and responses, or even just the request messages.
Hope the idea was of some help!
Regards,
A |
|
| Back to top |
|
|
|
|
