| Author |
Message
|
| Vitor |
 Posted: Mon Aug 03, 2015 11:13 am Post subject: |
 |
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| akil wrote: |
| Does this sound alright? |
How can we know? Also, it's for you to say.
I do repeat my misgivings about how much you're getting out of the pub/sub model here. You could just as easily:
1. create a mapping between the published message and the customer service
2. create a message flow that is MQInput -> (Compute of step 1 above) -> HTTPRequest
3. create a local queue for the customer
4. add a database entry for this new customer that includes the name of the local queue, and is used by the MQOutput node that replaces the Publication node in your current design.
I mean logically there's no real code difference between setting the topic of a Publication and setting the target queue in an MQOutput node. If you include the "customerxxx" in a standard queue name, you can probably get away without storing it in a database but just calculating it like a topic name.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Mon Aug 03, 2015 11:24 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
The only time you should be thinking about pub/sub is when you know - or at least strongly anticipate - needing more than one copy of each message.
It doesn't sound like you do. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Mon Aug 03, 2015 11:27 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| mqjeff wrote: |
The only time you should be thinking about pub/sub is when you know - or at least strongly anticipate - needing more than one copy of each message.
It doesn't sound like you do. |
Or when you are using something like MQLite which operates on a Pub/Sub model.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Aug 03, 2015 11:31 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| smdavies99 wrote: |
| mqjeff wrote: |
The only time you should be thinking about pub/sub is when you know - or at least strongly anticipate - needing more than one copy of each message.
It doesn't sound like you do. |
Or when you are using something like MQLite which operates on a Pub/Sub model. |
"Light". "Lite" is for water that pretends to be beer.
But, yes. If the actual transport you're using only supports pub/sub... then you obviously use pub/sub.
But plain MQ doesn't, and this is an IIB question that mentions MQInput nodes (which can't use MQLight... )
 |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Aug 03, 2015 11:39 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqjeff wrote: |
| The only time you should be thinking about pub/sub is when you know - or at least strongly anticipate - needing more than one copy of each message. |
Nice general rule
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| akil |
| Posted: Mon Aug 03, 2015 7:04 pm Post subject: |
|
|
Partisan
Joined: 27 May 2014
Posts: 338
Location: Mumbai
|
I thought this was a case for pub/sub because the message that is published goes to all subscribers ( hence multiple copies ) .
Subscribers only pick-up the messages that are of interest to them.
One way (I thought) was to use topics. I realise my mistake in this approach : if the topics are different for each subscriber, then there's really only 1 subscriber per topic, so this is essentially point-to-point, and not pub/sub.
The other way (I thought) could be to use a single topic , and use a content filter (Filter Node) inside the flow. Then there would be 1 topic for all subscribers and the flow would then looks like
MQInput --> Filter (match by customer) --> Compute --> HTTPRequest.
Since in MQInput I can't specify a topic, and i've to configure this in MQ as mentioned here http://www-01.ibm.com/support/knowledgecenter/SSFKSJ_7.5.0/com.ibm.mq.dev.doc/q026600_.htm, does it not mean that only 1 subscriber will get the message ( as the first one will remove it from the MQ ) ?
[/list][/list]
_________________
Regards |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Aug 04, 2015 4:19 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| akil wrote: |
The other way (I thought) could be to use a single topic , and use a content filter (Filter Node) inside the flow. Then there would be 1 topic for all subscribers and the flow would then looks like
MQInput --> Filter (match by customer) --> Compute --> HTTPRequest. |
If you've got 100+ customers, and the Filter node only results in true or false, how do you plan do make that work????
I would use a different mechanism than a Filter myself?
Given that you said:
| akil wrote: |
| there's really only 1 subscriber per topic, so this is essentially point-to-point |
then how is this a problem? 
Just because you got a single input, there's no reason that has to result in a single output. If, confusingly, there's now more than one subscriber for each inbound message then make a call for each customer than matches.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| akil |
| Posted: Tue Aug 04, 2015 5:37 am Post subject: |
|
|
Partisan
Joined: 27 May 2014
Posts: 338
Location: Mumbai
|
| Vitor wrote: |
| akil wrote: |
The other way (I thought) could be to use a single topic , and use a content filter (Filter Node) inside the flow. Then there would be 1 topic for all subscribers and the flow would then looks like
MQInput --> Filter (match by customer) --> Compute --> HTTPRequest. |
If you've got 100+ customers, and the Filter node only results in true or false, how do you plan do make that work????
|
MQInput --> Filter (match by customer) --> Compute --> HTTPRequest. is the subscribers flow, so each subscriber does something meaningful only when they get a message that interests them.
The publishers flow would simply be .... Compute -> Publication
I would use a different mechanism than a Filter myself?
| Vitor wrote: |
Given that you said:
| akil wrote: |
| there's really only 1 subscriber per topic, so this is essentially point-to-point |
then how is this a problem?
|
It is not a problem, i am trying to find out if I can avoid creating multiple local queues (one per customer)..
_________________
Regards |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Aug 04, 2015 6:00 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| akil wrote: |
| It is not a problem, i am trying to find out if I can avoid creating multiple local queues (one per customer).. |
The alternative is to build a flow that consumes the message, works out the customer it's for and then invokes the (apparently unique) Compute node to format the message, then sends it.
Pick your poison.
You certainly can't filter what the MQInput picks up.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Aug 04, 2015 6:33 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
The design question you're fundamentally dealing with is "How do I translate a message into the right format for a specific customer, and send it to the right location".
There is nothing about this problem that requires pub/sub.
There is nothing about this problem that requires one flow for each customer.
The most straight forward design is indeed something like - Receive internal message
- Determine customer destination
- Send to a specific flow for that customer
- Specific flow translates and sends message
But you don't need one flow for each customer.
You can use one flow that knows about all customers. This is a maintenance headache, because you have to test every customer in that flow each time any customer changes.
You can use one flow for each customer. This is a maintenance headache, because every time you add a new customer, you have to create a new queue and create a new flow and deploy and maintain it. You then have 100s of flows running, with hundreds of open queues with hundreds of threads with hundreds of chunks of memory...
Remember that your original flow has all of the information it needs to find out what customer the message belongs to. The second flow has all the information it needs to find out where to send the message for taht customer.
You can consider a balance between the two extremes of maintenance problems above. One flow can handle 5 customers or 10 customers. Each flow uses the same information that the main flow uses to find out what customer to use. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Aug 04, 2015 7:00 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| akil |
| Posted: Tue Aug 04, 2015 8:26 pm Post subject: |
|
|
Partisan
Joined: 27 May 2014
Posts: 338
Location: Mumbai
|
I've to learn to articulate it the way you did @mqjeff.. This are indeed the 2 extremes I am trying to balance out..
Testing the services repeatedly is a nightmare in this site, and that's the reason I was looking for suggestions towards 1 flow per customer.. I was trying to figure out an option that has the least resource consumption and lowest maintenance.
I looked at pub/sub as it seemed to provide one queue for all subscribers ( meeting the low maintenance requirement ).
Basis your suggestions, I'll think how to have 1 flow with low maintenance ( little testing for existing customers, when new customers are added ). I suppose XSLTransforms is an option - writing one XSL per customer, would not require me to test all customers as new XSL's are added..
_________________
Regards |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Aug 05, 2015 4:44 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| akil wrote: |
| I've to learn to articulate it the way you did @mqjeff.. |
He is good, isn't he?  
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Aug 05, 2015 5:01 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| akil wrote: |
| I suppose XSLTransforms is an option - writing one XSL per customer, would not require me to test all customers as new XSL's are added.. |
The transformation interface you use shouldn't affect your business requirements for testing - particularly for regression testing. So from the point of regression testing, ESQL should be the same as Java as .NET as PHP as XSLT.
Many shops would say that any change to a deployed application requires at least moderate regression testing of "unchanged" functions.
You should also take performance and statistics measurements of whatever combination of flows and queues and transformation interface you choose. |
|
| Back to top |
|
|
| akil |
| Posted: Wed Aug 05, 2015 5:47 am Post subject: |
|
|
Partisan
Joined: 27 May 2014
Posts: 338
Location: Mumbai
|
|
| Back to top |
|
|
|
|
