| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Sequential message processing across two Integration Nodes |
« View previous topic :: View next topic » |
| Author |
Message
|
| andrewfemin |
 Posted: Thu Mar 05, 2020 9:13 pm Post subject: Sequential message processing across two Integration Nodes |
 |
|
Acolyte
Joined: 26 Aug 2017
Posts: 54
|
Hi,
We have two linux servers running IIBv10 and they have the same set of EGs and message flows. This is done to increase the throughput and have better fault tolerant architecture. Each IIB server also has MQv9 from which the message flows read messages. Messages are sent to the queues in each IIB architecture using MQ cluster connection.
Now, in some of the message flows, there is a requirement for the messages to be processed in FIFO manner. When we have two servers processing messages parallely, sometimes one IIB server processes faster than the other and the second message ends up reaching the target before the first. We want to avoid this without shutting down the flow in one IIB server.
One of the ways we thought was to create the queues in a remote server and then have the flows in the IIB servers read the messages from the remote queues. And, have the queue manager release messages to IIB only after the previous message has been processed completely by IIB. For eg., message1 gets picked up by IIB1 and MQ waits till IIB1 has completed processing the message and then releases message2 to IIB2. Is this possible? We found this page in IBM KC which explains coordinated XA transactionality. Will this help achieve our purpose? Please help.
Thanks. |
|
| Back to top |
|
 |
| Vitor |
| Posted: Fri Mar 06, 2020 5:26 am Post subject: Re: Sequential message processing across two Integration Nod |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| andrewfemin wrote: |
| Now, in some of the message flows, there is a requirement for the messages to be processed in FIFO manner. |
This is called "message affinity" and is generally held to be a bad practice, mostly for the reasons you've discovered.
| andrewfemin wrote: |
| Is this possible? |
Of course it's possible. It's a recipe for disaster and you'd have to roll your own code to do it but you can write code that does anything. Certainly this is not possible:
| andrewfemin wrote: |
| Quote: |
| have the queue manager release messages to IIB only after the previous message has been processed completely by IIB. For eg., message1 gets picked up by IIB1 and MQ waits till IIB1 has completed processing the message and then releases message2 to IIB2. |
|
The queue manager will "release" a committed message as soon as an application asks for it. The best MQ will do is tell the application there's more than one message in a group and deliver them in sequence assuming the putting application added MQ group and sequence tags. It's incumbent upon your IIB code to figure out how to process all the messages, which is bad if you have message1 and message2. If you're talking about a true FIFO situation, this breaks down completely at message255 and higher (maximum number of messages in an MQ group) and your code will have a nervous breakdown at message10k.
This is also only work if the same queue manager is managing the entire group of messages, so there goes your cluster and fault tolerance.
| andrewfemin wrote: |
| We found this page in IBM KC which explains coordinated XA transactionality. Will this help achieve our purpose? |
No. This describes how to run a transaction across multiple points and does nothing to control the order in which actions are performed.
| andrewfemin wrote: |
| Please help. |
You have 2 options in my view.
Option 1 is to abandon your fault tolerant architecture (the cluster, the second server, all of it) and run all these FIFO messages through a single instance as I suspect they used to be run (this kind of requirement often arises from code that can't be bothered to maintain state so assumes that messages arrive in a defined order). You can design a manual failover so that processing moves from one IIB to the other in the event of a failure.
A variation on this is to design a single point of failure (database, queue manager, etc.) that's used by both instances for the kind of latchkey message release you describe. It's a question of how much you want to degrade your fault tolerance.
Option 2 is to rewrite the code so that message sequence is no longer important. 8 out of 10 developers will throw up there hands and claim that such a thing is impossible, but it's a straightforward coding exercise. If it was me (and obviously it isn't), I'd have both IIB instances share a global cache instance (which comes free with IIBv10), have code that receives message 1 add it to the cache & stop, have code that retrieves message1 when it receives message2 and processes them both.
For the code instances that need true FIFO and must process message1 - messageN in order, I would beat the developer round the head with the Trout Of Teaching while pointing out that in a normal single instance MQ topology there's no guarantee that MQ messages will arrive in the order that they are sent and to assume such a thing in code is a bad idea. Not least of which is you can't add additional instances to the flow (because the thread processing messageN+1 might complete before the one processing messageN) and not being able to scale a flow is worse than a fault intolerent flow.
Once they have accepted their sin, I would send them off to redesign their flow. If there's pushback from management on the additional time & resources needed, I would use the Trout of Management Expectations on them.
(It's like the Trout Of Teaching but has a housebrick in it, and my expectation is that management will stop pushing back and start falling down).
Happy Friday to all forum regulars.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Mar 06, 2020 5:40 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
What my esteemed colleagues where aiming at is that you would need to open the target cluster queue not with bindings not fixed but bindings group. This of course if your affinity is reduced to a group level. If your affinity is truly across all messages, review the design!!! 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| andrewfemin |
| Posted: Sat Mar 07, 2020 9:58 pm Post subject: Re: Sequential message processing across two Integration Nod |
|
|
Acolyte
Joined: 26 Aug 2017
Posts: 54
|
Hello,
Thanks for replying.
| Vitor wrote: |
| andrewfemin wrote: |
| Is this possible? |
Of course it's possible. It's a recipe for disaster and you'd have to roll your own code to do it but you can write code that does anything. |
Apologies for not being clear. I was trying to avoid any code level changes and trying to see if this is possible only with MQ. There is a lot of legacy code involved.
| Vitor wrote: |
Option 1 is to abandon your fault tolerant architecture (the cluster, the second server, all of it) and run all these FIFO messages through a single instance as I suspect they used to be run (this kind of requirement often arises from code that can't be bothered to maintain state so assumes that messages arrive in a defined order). You can design a manual failover so that processing moves from one IIB to the other in the event of a failure. |
You're right about the earlier state being a single instance. We recently upgraded the architecture to a Active-Active mode and that is when these issues started.
| Vitor wrote: |
...while pointing out that in a normal single instance MQ topology there's no guarantee that MQ messages will arrive in the order that they are sent and to assume such a thing in code is a bad idea. |
Now, why would you say that? In a normal single instance MQ topology, don't the messages get processed in FIFO except in case of message priority sequencing?
Thanks,
Andrew |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sun Mar 08, 2020 7:49 am Post subject: Re: Sequential message processing across two Integration Nod |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| andrewfemin wrote: |
| Vitor wrote: |
...while pointing out that in a normal single instance MQ topology there's no guarantee that MQ messages will arrive in the order that they are sent and to assume such a thing in code is a bad idea. |
Now, why would you say that? In a normal single instance MQ topology, don't the messages get processed in FIFO except in case of message priority sequencing?
|
When messages expire, when queues fill, when messages go to dead letter queues, when messages gets backed out, when messages get offloaded to backout queues, all these are potential opportunities for the order of the messages to be altered even when dealing with a single queue.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Mon Mar 09, 2020 2:49 pm Post subject: Re: Sequential message processing across two Integration Nod |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| andrewfemin wrote: |
| Vitor wrote: |
| andrewfemin wrote: |
| Is this possible? |
Of course it's possible. It's a recipe for disaster and you'd have to roll your own code to do it but you can write code that does anything. |
Apologies for not being clear. I was trying to avoid any code level changes and trying to see if this is possible only with MQ. There is a lot of legacy code involved. |
If there is legacy code and app design involved, your only option is to run a single instance of the flow. Been there, forced into doing that.
| Vitor wrote: |
...while pointing out that in a normal single instance MQ topology there's no guarantee that MQ messages will arrive in the order that they are sent and to assume such a thing in code is a bad idea. |
| andrewfemin wrote: |
| Now, why would you say that? In a normal single instance MQ topology, don't the messages get processed in FIFO except in case of message priority sequencing? |
Vitor is saying that because its true. We always tell designers that MQ message order is not guaranteed, and it must never be relied on for MQ messaging solutions. Search for MQ message marshalling on this forum.
_________________
Glenn |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
