| Author |
Message
|
| JohnSmith |
 Posted: Thu Oct 28, 2010 3:00 am Post subject: Websphere Message Broker: A design Question |
 |
|
Voyager
Joined: 17 Mar 2010
Posts: 86
|
Hi All,
we are having a main Routing flow which will do the routing of the original message to 95 different OPERATION flows.
We are using MQ protocol here, where the Main flow would output the message to a QUEUE(which will be different for each flows) and the corresponding operation flow(which is listening to this particular queue would pick up the message).
The only approach seems to be is to use MQInput nodes for the entry points into each of 95 flow with different queue names.
But, this process requires a creation of 95 different local queues, isn't there a better solution for this? I couldn't come up with one, can anyone suggest something.
Thanks in advance for your help. |
|
| Back to top |
|
 |
| yaemu |
| Posted: Thu Oct 28, 2010 3:17 am Post subject: |
|
|
Novice
Joined: 27 Oct 2010
Posts: 17
|
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Oct 28, 2010 3:23 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Why is it a problem to have 95 queues?
How complicated is each operation? How much does each operation relate to the other?
If you bundle all 95 operations in one flow using RouteToLabel, how much do you have to test if you change one of those operations? |
|
| Back to top |
|
|
| yaemu |
| Posted: Thu Oct 28, 2010 3:50 am Post subject: |
|
|
Novice
Joined: 27 Oct 2010
Posts: 17
|
MQJiff,
Your reply makes me ask, when to use the RouteToLabel and when to not use it?
I really need to know. because I am using the RouteToLabel node too much. |
|
| Back to top |
|
|
| JohnSmith |
| Posted: Thu Oct 28, 2010 4:07 am Post subject: |
|
|
Voyager
Joined: 17 Mar 2010
Posts: 86
|
Hi mqjeff and yaemu, thanks for your reply.
we thought about the Label node approach earlier but in that case we need to make everything part of a single flow which we do not want because of many reasons regarding other design constraints of our project. Those 95 flows should be separate.
mqjeff wrote:
| Quote: |
| How complicated is each operation? How much does each operation relate to the other? |
Well, each operation is different from other in many ways and complexity varies from simple to highly complex.
what would be great is to have the main flow and operation flow separate at the same time save time on logistics part like creation of 95 queues which would require maintenance as well.
Is it possible to reduce the number of queues? Can we somehow use MQInput node using only Browse option? |
|
| Back to top |
|
|
| joebuckeye |
| Posted: Thu Oct 28, 2010 4:41 am Post subject: |
|
|
Partisan
Joined: 24 Aug 2007
Posts: 365
Location: Columbus, OH
|
| Quote: |
| Those 95 flows should be separate. |
If this is a requirement then how can you keep them separate but not use queues to communicate? You would need to use some other form of input node. HTTP? SOAP? FileInput?
If you don't want to have all 95 operations as subflows of the routing flow then these operation flows have to start with something. Having them all as subflows would not be a good design based on what you said earlier as it sounds like some of these flows will take larger to process than others and being able to give some operation flows more instances to handle load is possible if you keep the operation flows separate and independent of the routing flow.
And 95 queues is not a big deal. Your MQ admin team builds a script to create these queues in a test environment and then tweak it based on any errors your testing of the 95 flows finds so that when they go to build this in production everything is correct.[/i] |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Oct 28, 2010 5:05 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
A new cleaning product perhaps? 
If you're going to ask for advice, at least spell our names right.
| yaemu wrote: |
| Your reply makes me ask, when to use the RouteToLabel and when to not use it? |
What my most worthy associate seems to be getting at is that labels have to be in the same flow as their route, i.e. subflows. Hence they become part of the flow at deploy time increasing size and making it impossible to split them out into different run time components. This also (as he correctly points out) means that a change to such a sub flow requires the redeploy (and therefore the retest) of the entire flow.
Labels are good for "sub functions" within a single flow. An exmaple: I have a flow at the moment that generates statisitical information from XML input for a mainframe system. The first part of the flow sets up the boilerplate for the record (ids, effective dates, etc) and then shoots off via labels to perform specific processing for specific lines of business.
Where you're simply trying to invoke unrelated functions from a single point (as here) that's unwieldy.
| yaemu wrote: |
| I really need to know. because I am using the RouteToLabel node too much. |
It's only too much if you use it to the exclusion of all other solutions.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Oct 28, 2010 5:07 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| joebuckeye wrote: |
| And 95 queues is not a big deal. Your MQ admin team builds a script to create these queues in a test environment and then tweak it based on any errors your testing of the 95 flows finds so that when they go to build this in production everything is correct. |
A couple of hundred queues on a queue manager is not uncommon. My personal record is 5000-odd queues on a single queue manager (which was a little excessive I felt, but the application team insisted!).
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| vikas.bhu |
| Posted: Fri Oct 29, 2010 1:04 am Post subject: |
|
|
Disciple
Joined: 17 May 2009
Posts: 159
|
Going back to question..
Why not to generate the 95 request format in WMB and and call the 95 applications directly.
it will reduce the no of queues and and improve performance.
(use the queues if required. and the call the application whome you can call directly).
If u expalin more then we can come back with better solution.(may be) |
|
| Back to top |
|
|
| vmcgloin |
| Posted: Fri Oct 29, 2010 2:53 am Post subject: |
|
|
Knight
Joined: 04 Apr 2002
Posts: 560
Location: Scotland
|
Can you expand on the requirement to have a single routing flow?
Why is this a requirement? Do the operations fall into service groups that could have their own end-points, and then use routeToLabel at the appropriate level within those?
Having asked those question, historically at my company they have used massive routing flows (with routeToLabel and route to queue) and these are currently being rationalised to route via a queue. i.e. subflows are being promoted to 'secondary' flows. This allows independent scaling, management and monitoring.
As othere have said, 95 queues is not a large number (does that include reply flows?). Just ensure your naming standards are followed and you are happy with your release processes. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Oct 29, 2010 2:56 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| vikas.bhu wrote: |
| Why not to generate the 95 request format in WMB and and call the 95 applications directly. |
So eliminate WMQ and all the advantages it offers, plus requireing the complete retest of the applications if there's a single change?
Big price to pay to eliminate some queues. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| joebuckeye |
| Posted: Fri Oct 29, 2010 4:24 am Post subject: |
|
|
Partisan
Joined: 24 Aug 2007
Posts: 365
Location: Columbus, OH
|
We have a service routing flow that accepts XML message with our own message header in it. This message header contains service name and service version (among other things). All the requests come to this flow.
The flow then looks up the service name and service version in a database to collect service information.
This service information contains how to handle this request. Right now we can either send the message to MQ, HTTP or to publish the message. We have over 100 services listed in the database but since we have these delivery type patterns we simply store the QM/Q for MQ services, the URL for HTTP requests and the topic for publications.
For the MQ type we set up the destination list in the message so that a single MQ Output node handles all the services. This also allows us to add new services that don't require a code change (unless they need new functionality not provided by the basic flow).
We use the route to label node to drive the request to the correct service type (MQ, HTTP or publishing) but not to individual queues. Use the route to label to separate out different functionality types, not individual destinations. |
|
| Back to top |
|
|
| BDS |
| Posted: Tue Nov 02, 2010 6:09 am Post subject: |
|
|
Newbie
Joined: 21 Oct 2010
Posts: 8
|
Personally I'd use RTL for ony a handful of destinations for ease of reading only. 95 labels would be an ugly vision in the toolkit and it is easier to do a find in the esql for the content based routing logic.
Previously, I have dynamically built the destination queue based on content in the message rather than have the db look up cost (you could also consider caching the rules). The CBR logic is hosted in a common package and resued for all messages in the environment - if it changes, you can change it in one place and helps the regression testing effort for future changes
If you have a volatile routing reqns environment, this approach works well in a pub/sub pattern - you can add a new operation flow very easilily based on the dynamic topic string built in the common logic.
The benefit of having 95 operation queues (you might be able to reduce this is the flow supports a business service/application or function and perform further content based routing) is that you can tune the number of instances based on your usage profile. useful to have several flows for admin purposes for maint windows too. |
|
| Back to top |
|
|
|
|
