| Author |
Message
|
| akil |
 Posted: Mon Aug 03, 2015 8:50 am Post subject: IIB9: Pub/Sub |
 |
|
Partisan
Joined: 27 May 2014
Posts: 338
Location: Mumbai
|
I've to create a publisher flow, and multiple (4+) subscriber flows.
From earlier posts in this forum, and the samples/documentation, I understand that to 'publish' to topics, the 'publication' node can be used. Unlike the MQ Output node, however, the 'Publication' node has no properties to specify the MQ Manager Name, so
Question 1: Which Q Manager will the 'Publication' node use - is it always the QManager that is tied to the Broker?
Question 2: Is it possible to specify the Q Manager to be a different one?
To create a subscriber flow, I've seen the 'topic' property to be available for JMSInput, however the 'topic' property of MQInput seems to say that the topic is 'assigned' to the messages that are read, and not 'filter' . I could not find samples / documentation on creating a subscriber app within the broker.. Any hints on how to do this?
_________________
Regards |
|
| Back to top |
|
 |
| Vitor |
| Posted: Mon Aug 03, 2015 9:13 am Post subject: Re: IIB9: Pub/Sub |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| akil wrote: |
| Question 1: Which Q Manager will the 'Publication' node use - is it always the QManager that is tied to the Broker? |
IIBv9 always and only uses the queue manager that is tied to the broker.
| akil wrote: |
| Question 2: Is it possible to specify the Q Manager to be a different one? |
IIBv9 always and only uses the queue manager that is tied to the broker.
One of the selling points of the publish/subscribe model is that it fully decouples the producer of the data (publisher) and the consumer of the data (subscriber). It's rather anti-pattern to have a requirement to publish to a specific queue manager; if that is the requirement, you might be better with a more conventional point-to-point model.
| akil wrote: |
| I could not find samples / documentation on creating a subscriber app within the broker.. Any hints on how to do this? |
The simplest way is to manually create a subscription with the filters and so forth that you want and point it at a local queue. Then consume what gets to that queue with an MQInput node.
Other and potentially better methods are undoubtedly possible.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Aug 03, 2015 9:21 am Post subject: Re: IIB9: Pub/Sub |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| akil wrote: |
| Unlike the MQ Output node, however, the 'Publication' node has no properties to specify the MQ Manager Name |
To be clear, and for the benefit of future readers, setting the queue manager name in the MQOutput node doesn't affect the actual put of the message. The message is always put to the queue manager associated with the broker; if you set this property on the node, it sets the target queue manager name within WMQ, causing the broker's queue manager to route the message in accordance with the WMQ name resolution rules.
 to the original post, but I thought it was worth being explicit.
The ability to use a queue manager other than the one associated with the broker is one of the big new features of IIBv10. IMHO it's such a structural change it's unlikely to be ported back into IIBv9 through a fix pack or similar.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| akil |
| Posted: Mon Aug 03, 2015 9:35 am Post subject: |
|
|
Partisan
Joined: 27 May 2014
Posts: 338
Location: Mumbai
|
| Quote: |
| The simplest way is to manually create a subscription with the filters and so forth that you want and point it at a local queue. Then consume what gets to that queue with an MQInput node. |
How do I manually create a subscription ? Do you mean on the MQ Explorer ? I suppose this is called the 'unmanaged subscriber' in the following link - http://www-01.ibm.com/support/knowledgecenter/SSFKSJ_7.5.0/com.ibm.mq.dev.doc/q026590_.htm
_________________
Regards |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Aug 03, 2015 9:43 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| akil wrote: |
| How do I manually create a subscription ? Do you mean on the MQ Explorer ? |
Or through script, or however you create local queues and other objects on your site. It's just another object.
You suppose wrong. The unmanaged subscriber in that link is using the MQSUB verb to extend an administrative subscription; IIB cannot do that. IIB is the MQ Publication Consumer in that link.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| akil |
| Posted: Mon Aug 03, 2015 9:50 am Post subject: |
|
|
Partisan
Joined: 27 May 2014
Posts: 338
Location: Mumbai
|
Thank you, understood. One follow up question,
If i've to create 5+ subscribers, all on the same broker, does this mean that I have to create a separate local queue per subscriber ( and keep adding a local queue everytime a new subscriber gets added ) , or is there any alternative with a little less management overhead (aka single local queue with topics)..
_________________
Regards |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Aug 03, 2015 10:08 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| akil wrote: |
| If i've to create 5+ subscribers, all on the same broker, does this mean that I have to create a separate local queue per subscriber ( and keep adding a local queue everytime a new subscriber gets added ) |
If your requirement is that all 5+ subscribers require their own copy of the publication then you need 5+ local queues to store their copies. Each new distinct subscriber requires a new queue.
It's possible to have one subscription (e.g. /Fruit) and use logic inside the consuming flow to pass the message to different flow or sub-flow (e.g. /Fruit/Apple goes one way, /Fruit/Orange goes another). There are a number of dynamic routing options inside IIB that would be suitable.
If you have subscribers that subscribe to /Fruit/Apple, /Fruit/Orange, /Vegetable/Carrot, etc then clearly this model is no use.
| akil wrote: |
| or is there any alternative with a little less management overhead (aka single local queue with topics).. |
A single local queue only holds messages. The topic (and subscription) controls routing and delivery. One (of I'm sure many alternatives) would be to use the JMSInput node and apply a selector to the messages in the queue, or indeed configure the node to subscribe directly. Is the management overhead of the JNDI and other JMS connection requirements better or worse than the management of a durable subscription and a local queue? You need to make that determination based on the details of your requirements; no-one here can do that because we don't know. Points to consider include but are not limited to:
- how many new subscribers are there likely to be, and how frequently?
- what is supposed to happen if a subscriber is down at time of publication?
- how many, if any, non-IIB subscribers will there be? Now and in the future?
- how many, if any, non-IIB publishers will there be? Now and in the future?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| akil |
| Posted: Mon Aug 03, 2015 10:17 am Post subject: |
|
|
Partisan
Joined: 27 May 2014
Posts: 338
Location: Mumbai
|
Thank you, I'll think about this some more.
A few things that I am aware of already:
This starts off with 5 subscribers.
There will be many subscribers, with a couple getting added every few weeks, totalling to around 100 in a year.
The subscriber flow will simply be ,
MQInput/JMSInput --> Compute --> HTTPRequest ( essentially notifying a non-IIB web-service that is interested in consuming events )..
The events get published as new incoming messages are received from payment networks. And subscribers are interested in only specific events (funds hitting their respective accounts for eg)..
_________________
Regards |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Aug 03, 2015 10:24 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| akil wrote: |
| The events get published as new incoming messages are received from payment networks. And subscribers are interested in only specific events (funds hitting their respective accounts for eg).. |
I'd think hard about using a pub/sub model. Accepting that the input is an incoming message over WMQ, and that the inbound message needs to be routed to 1-100 web-services that are "subscribed" to those events, I'm not convinced that using WMQ pub/sub is actually getting you much in this topology. It seems that you could funnel all the inbound messages to a single queue, then route & filter them using IIB facilities to the correct web-service. The IIB Business Decision Service might be a good and flexible way to implement that sort of thing.
But like I said, it's your call at the end of the day.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Aug 03, 2015 10:27 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Vitor wrote: |
| If your requirement is that all 5+ subscribers require their own copy of the publication then you need 5+ local queues to store their copies. Each new distinct subscriber requires a new queue. |
Every subscription receives a copy of every message published to the topic of interest.
Whether or not each subscription requires it's own queue is a design point for the consuming application.
With a subscriber flow like
| Quote: |
| MQInput/JMSInput --> Compute --> HTTPRequest |
it could be possible that each message received has enough information to determine the URL for the HTTPRequest, and therefore only one queue/JMS Destination is required. Many instances of the flow might be needed, but. |
|
| Back to top |
|
|
| akil |
| Posted: Mon Aug 03, 2015 10:41 am Post subject: |
|
|
Partisan
Joined: 27 May 2014
Posts: 338
Location: Mumbai
|
With a pub/sub mode, as new consumers are added, only new message flows need to be deployed, the publishing application (which is another flow on the broker) isn't required to be changed.. That's one advantage..
@mqjeff, if there's a single queue, wouldn't just one of the 100+ flows read the message? How would a single queue work in this case?
_________________
Regards |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Aug 03, 2015 10:52 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| akil wrote: |
| @mqjeff, if there's a single queue, wouldn't just one of the 100+ flows read the message? How would a single queue work in this case? |
So, the question is, why do you need 100+ flows? That is, you probably don't.
If each publication contains enough information that you can figure out what the URL to call from the HTTPRequest node is, then you only need 100+ subscriptions, not 100+ flows. |
|
| Back to top |
|
|
| akil |
| Posted: Mon Aug 03, 2015 10:53 am Post subject: |
|
|
Partisan
Joined: 27 May 2014
Posts: 338
Location: Mumbai
|
The Compute node would be different for the different services, as each service has it's own specific message, which is to be created/populated with the details of the incoming payment .
_________________
Regards |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Aug 03, 2015 10:59 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| akil wrote: |
| The Compute node would be different for the different services, as each service has it's own specific message, which is to be created/populated with the details of the incoming payment . |
So each service has a message so distinct from all the other services that there's absolutely no chance of commonality and re-use? The only way to achieve this is for each subscriber to have it's own absolutely distinct flow?
I withdraw my comment about using JMSInput nodes. That's a lot of JNDI and with so many nearly identical topics you'll benefit from using the durable subscriptions to track messages.
Also the management overhead of tracking 100+ administrative changes to the queue manager will be as nothing compared to tracking 100 highly similar but absolutely distinct Compute nodes and their processing while aligning them with the subscriptions that they service! 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| akil |
| Posted: Mon Aug 03, 2015 11:06 am Post subject: |
|
|
Partisan
Joined: 27 May 2014
Posts: 338
Location: Mumbai
|
These services are 'customer' provided, and yes, they have little in common (no standards here). As such, when each 'customer' is on-boarded to receive these events, I've to
1. create a mapping between the published message and the customer service
2. create a message flow that is MQInput -> (Compute of step 1 above) -> HTTPRequest
3. create a topic for the customer '/notifications/customerxxx'
4. create a local queue for the customer
5. route the subscription to the local queue
6. add a database entry for this new customer
The single publisher flow, will publish messages for all customers that have a database entry ..
Does this sound alright?
_________________
Regards |
|
| Back to top |
|
|
|
|
