| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Pub/Sub in an MQ Cluster with MQ Clients |
« View previous topic :: View next topic » |
| Author |
Message
|
| PeterPotkay |
 Posted: Sun Jan 17, 2010 8:14 am Post subject: Pub/Sub in an MQ Cluster with MQ Clients |
 |
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
The population of subscriber apps will be small (3-5) and static (no plans right now for any new subscribers). We will be using predefined subscription queues so that we can provide alerting and historical monitoring. The application has a requirement that its subscribers do not miss any publications, so the subscriptions will be Durable. The application understands that any new subscribers, including the initial ones, will only get new publications, so the publications will be Non-Retained.
There will be 3 Queue Managers in this MQ Pub/Sub cluster, and each QM will in turn be Highly Available using traditional hardware clustering.
Everything will be MQ 7.0.1 or newer.
The Publishers and Subscribers will all be MQ Clients connecting from servers other than the ones hosting these QMs. The plan is to use the load balancing feature of the MQ 7 Client Channel tables for both Publishers and Subscribers.
When a Publisher connects, it doesn't care which of the 3 QMs it connects to. Which ever QM it does connect to will happily accept its publications and make them available to the other 2 QMs. If the connected-to QM goes down and takes a minute to fail over from Node 1 to Node 2, the Publisher is free to connect to one of the other 2 QMs and continue to publish.
Here is my question:
How are the subscribers going to insure they don't miss any messages, even with Durable subscriptions, in the event the QM they are connected to goes down for a minute as it fails over? If the subscriber is initially connected to QM1, and consuming its publications as they come in from a Publisher connected to QMx, what happens if QM1 goes down during a burst of heavy publication activity? The subscriber can reconnect and find itself connected to QM2 or QM3, (resubscribe?) and I assume new publications will start going to QM2 or QM3, but I see the potential of marooned messages on QM1. QM1 will come up in a minute or so on its alternate node, but the subscriber has now connected to QM2 or QM3 and is getting its new messages there. There might be a few messages left over on QM1. And I'm guessing the Durable subscription is still valid on QM1, so are there going to be Publications now going to the "orphaned" QM1 subscription AND to the new one on QM2/3?
How does one insure a subscriber does not miss any messages in this scenario? Or is it that in a Pub Sub cluster a subscriber should only ever keep a subscription on one of the QM's in the cluster, and not take advantage of channel tables to reconnect to an alternate QM?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
 |
| mvic |
| Posted: Sun Jan 17, 2010 1:18 pm Post subject: Re: Pub/Sub in an MQ Cluster with MQ Clients |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| PeterPotkay wrote: |
| How does one insure a subscriber does not miss any messages in this scenario? Or is it that in a Pub Sub cluster a subscriber should only ever keep a subscription on one of the QM's in the cluster, and not take advantage of channel tables to reconnect to an alternate QM? |
If you're a subscriber who connects and subscribes on QM1, then later reconnects after a failure of a connection to QM1, why would there be any messages for you on QM2? I may not have a complete enough understanding of pub/sub clustering, but the fundamentals of MQ apply here - a message arrives for you onto a queue. There is still (today, on distributed platforms anyway) no ability to connect to QM2 and obtain messages that were put on your queue on QM1. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sun Jan 17, 2010 3:59 pm Post subject: Re: Pub/Sub in an MQ Cluster with MQ Clients |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| mvic wrote: |
| PeterPotkay wrote: |
| How does one insure a subscriber does not miss any messages in this scenario? Or is it that in a Pub Sub cluster a subscriber should only ever keep a subscription on one of the QM's in the cluster, and not take advantage of channel tables to reconnect to an alternate QM? |
If you're a subscriber who connects and subscribes on QM1, then later reconnects after a failure of a connection to QM1, why would there be any messages for you on QM2? I may not have a complete enough understanding of pub/sub clustering, but the fundamentals of MQ apply here - a message arrives for you onto a queue. There is still (today, on distributed platforms anyway) no ability to connect to QM2 and obtain messages that were put on your queue on QM1. |
I understand that the client can't get the marooned messages from QM1 via QM2. That's a fact of life for pub sub or non pub sub MQ clustering. (z/OS Shared Queueing is not an option here.)
My question is, "In a Pub Sub cluster, can the subscribers use MQ Client channel tables to connect to alternate QMs if the QM they connected to originally and subscribed to originally goes down, either for a 1 minute fail over in a hardware cluster or for several hours due to a more severe error."
I think not because using Durable Subscriptions (remember, the requirement is the subscriber needs all relevant publications) you risk marooned publications on QM1 and then duplicate messages once QM1 comes back but after the subscriber has resubscribed on QMx.
Unless someone can explain otherwise, I think a particular durable subscriber in a Pub Sub cluster can only connect to one QM* and so those QMs better be H.A. A Pub Sub cluster can really help on the Publication side, by allowing the publishers a range of QMs to connect to. And they can help for load balancing, by allowing some subscribers to use QM1, some to use QM2 and some to use QM3. But for any individual subscriber, there is no benefit to a Pub Sub Cluster if they don't want to miss any messages or duplicate any messages when they reconnect to QMx?
*They are free of course to connect to another QM in the cluster if they unsubscribe from QM1 during a period of no publications and resubscribe on QMx prior to the resuming of publication.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mvic |
| Posted: Sun Jan 17, 2010 4:14 pm Post subject: Re: Pub/Sub in an MQ Cluster with MQ Clients |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| PeterPotkay wrote: |
| But for any individual subscriber, there is no benefit to a Pub Sub Cluster if they don't want to miss any messages or duplicate any messages when they reconnect to QMx? |
I agree with the essence of your proposition. A durable subscriber has the same affinity with the physical location of its durable subscription as a traditional getter has with its queue.
That's the way I understand it anyway.
(There IS a benefit to a subscriber indirectly, from the improvement in quality of service gained by publishers...) |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
