MQSeries.net :: View topic - No Single Point of Failure and no dupe msgs pub/sub clusters

simonalexander2005 · Posted: Tue Jul 13, 2021 6:26 am Post subject:

I have a cluster of 3 queue managers (latest MQ on linux), each copies of each other, for HA purposes.

The publishing application writes to a clustered alias queue that points to a topic, with cluster priorities set to retain message sequencing (i.e. all messages get sent to the same queue manager unless it goes down).

The queue managers are Topic Hosts (i.e. we are using a single clustered topic across all 3 queue managers, per message type).

I now need to come up with a resilient design for applications which need to receive these messages. The following three scenarios need to be accounted for:

a) coded applications that connect directly into the cluster to get the message (we can say exactly how they get the message - e.g. from a queue which we provide, or via subscriptions we provide, or via their own subscriptions)
b) applications that read from queues on the customer's own queue manager, or set of queue managers, which can be brought into our cluster
c) applications that read from queues on the customer's own queue manager, or set of queue managers, which can not be brought into our cluster

From what I understand from the documentation, scenario a) is simple enough - the application can create subscriptions on all of my queue managers, subscribing directly to the clustered topic, and use subscription grouping to ensure they don't receive duplicate messages. Is my understanding here correct?

For scenarios b) and c), we need to push the messages out to the client queue managers for their application to then read from - but we only want to send one copy of the message into their system, rather than one copy per queue manager; to prevent duplicate messages being processed on their end. But if we only subscribe in one place, we're losing resilience.

My hope was to be able to do something similar to scenario a) - create a subscription on each of our queue managers pointing at an alias with cluster priorities on their queue manager (via remote queues if they can't be brought into our cluster) - but of course, according to the documentation, I can't use subscription grouping on managed subscriptions - so they would get one copy of the message per subscription which is not what we want.

In scenario b) they could instead subscribe directly to the topics (rather than us creating the subscriptions on our qmgrs) - this allows them to receive one copy of the message per queue manager they have, which might be ok? But it's not an ideal scenario if I can avoid it.

What other options do I have?

If the topics weren't clustered - i.e. they were only locally defined on each of the three queue managers, then the issue would go away I think, because the publisher would only ever be publishing to one of the topics at once; and so I could just have subscriptions across each of my three queue managers, and whichever topic is processing the messages, the subscription on the same queue manager would pick up and send to a clustered alias queue on their side (via remote/xmit queues, in the case of scenario c)).

However, where clustered topics are used, and there are are multiple receiving queue managers, how can I go about getting the messages onto their receiving clustered alias queue without duplicating the messages and without losing resilience in the case of failure on our side? I want some kind of subscription grouping that works for managed subscriptions; but that appears to not be an option.