ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » Clustering » No Single Point of Failure and no dupe msgs pub/sub clusters

Post new topic  Reply to topic
 No Single Point of Failure and no dupe msgs pub/sub clusters « View previous topic :: View next topic » 
Author Message
simonalexander2005
PostPosted: Thu Jul 08, 2021 3:42 am    Post subject: No Single Point of Failure and no dupe msgs pub/sub clusters Reply with quote

Acolyte

Joined: 13 Jun 2016
Posts: 55

I have a cluster of 3 queue managers (latest MQ on linux), each copies of each other, for HA purposes.

The publishing application writes to a clustered alias queue that points to a topic, with cluster priorities set to retain message sequencing (i.e. all messages get sent to the same queue manager unless it goes down).

The queue managers are Topic Hosts (i.e. we are using a single clustered topic across all 3 queue managers, per message type).

I now need to come up with a resilient design for applications which need to receive these messages. The following three scenarios need to be accounted for:

a) coded applications that connect directly into the cluster to get the message (we can say exactly how they get the message - e.g. from a queue which we provide, or via subscriptions we provide, or via their own subscriptions)
b) applications that read from queues on the customer's own queue manager, or set of queue managers, which can be brought into our cluster
c) applications that read from queues on the customer's own queue manager, or set of queue managers, which can not be brought into our cluster

From what I understand from the documentation, scenario a) is simple enough - the application can create subscriptions on all of my queue managers, subscribing directly to the clustered topic, and use subscription grouping to ensure they don't receive duplicate messages. Is my understanding here correct?

For scenarios b) and c), we need to push the messages out to the client queue managers for their application to then read from - but we only want to send one copy of the message into their system, rather than one copy per queue manager; to prevent duplicate messages being processed on their end. But if we only subscribe in one place, we're losing resilience.

My hope was to be able to do something similar to scenario a) - create a subscription on each of our queue managers pointing at an alias with cluster priorities on their queue manager (via remote queues if they can't be brought into our cluster) - but of course, according to the documentation, I can't use subscription grouping on managed subscriptions - so they would get one copy of the message per subscription which is not what we want.

In scenario b) they could instead subscribe directly to the topics (rather than us creating the subscriptions on our qmgrs) - this allows them to receive one copy of the message per queue manager they have, which might be ok? But it's not an ideal scenario if I can avoid it.

What other options do I have?

If the topics weren't clustered - i.e. they were only locally defined on each of the three queue managers, then the issue would go away I think, because the publisher would only ever be publishing to one of the topics at once; and so I could just have subscriptions across each of my three queue managers, and whichever topic is processing the messages, the subscription on the same queue manager would pick up and send to a clustered alias queue on their side (via remote/xmit queues, in the case of scenario c)).

However, where clustered topics are used, and there are are multiple receiving queue managers, how can I go about getting the messages onto their receiving clustered alias queue without duplicating the messages and without losing resilience in the case of failure on our side? I want some kind of subscription grouping that works for managed subscriptions; but that appears to not be an option.
Back to top
View user's profile Send private message
simonalexander2005
PostPosted: Tue Jul 13, 2021 6:26 am    Post subject: Reply with quote

Acolyte

Joined: 13 Jun 2016
Posts: 55

I have since learned that Subscription Grouping only applies where you have different topic strings (which might overlap), and not where you have multiple subscriptions to the same topic string from multiple instances of an application.

So now none of my solutions lead to deduplicated subscriptions...
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » Clustering » No Single Point of Failure and no dupe msgs pub/sub clusters
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.