| Author |
Message
|
| nelson |
 Posted: Wed Jul 15, 2015 8:22 am Post subject: Broker calling itself |
 |
|
Partisan
Joined: 02 Oct 2012
Posts: 313
|
Hi all,
Is it a bad idea to call a service within the broker, from the broker? Some people say it is a bad practice... what do you think? Something is bad in your design when the broker needs to call itself, to build a more complex "service" (maybe a "process")? Are relaying in this case some orchestrating activities that broker shouldn't take care off?
Please notice that I'm talking about what we usually call "services", not this kind of reusable or common flows, but about a particular business need that we translate in a "service" within broker.
Appreciate your inputs.
Kind regards. |
|
| Back to top |
|
 |
| Vitor |
| Posted: Wed Jul 15, 2015 8:38 am Post subject: Re: Broker calling itself |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| nelson wrote: |
| Some people say it is a bad practice... what do you think? |
Who are these "people"? Within your organization, or publically? If the latter, post links (let's get all the views in one thread.
| nelson wrote: |
| Please notice that I'm talking about what we usually call "services", not this kind of reusable or common flows, but about a particular business need that we translate in a "service" within broker. |
So you're asking about a SOAPRequest node calling a service that's provided by a flow with a SOAPInput node? I'll assume so for the purposes.
| nelson wrote: |
| Something is bad in your design when the broker needs to call itself, to build a more complex "service" (maybe a "process")? Are relaying in this case some orchestrating activities that broker shouldn't take care off? |
My 2 cents only:
It's perfectly acceptable and desirable for a broker flow to call out to an existing service where that service does something the flow logic needs rather than reproduce that service in-line to the flow. It doesn't make any realistic difference if that service is hosted in WAS or in WMB/IIB. All the usual considerations for a multi-customer web service (provides a broad based service, maintains backwards compatibility or versioning, etc, etc) apply to a broker hosted web service as they do for any other. If the service is not that broad based than it probably should be a sub flow or other internal construct, but that's a case by case design decision.
What you shouldn't have inside broker is a process, especially a user driven or "long running" process. Broker is an integration engine and should not maintain state. Now again it's a design decision when a flow becomes a process, but in my view the fact that a flow has to call multiple services doesn't make it so. Broker certainly shouldn't be orchestrating in the way you describe, and if you're calling out to multiple services the design needs to account for that (probably using asynchronous techniques). If the "service" needs to be orchestrated it's a process and is better placed in Process Server or other such technologies.
So no, it's not an issue if broker is calling to itself. The issue is if the broker integration flow has so many calls it's morphed into a process or a business application.
Other views are equally valid and may be better.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Wed Jul 15, 2015 9:49 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
If you followed the books on 3-tier ESB's you would be doing this all the time
A business level service would call one or more business services or technical service and these could call component services.
All implemented on the same broker as well!
However this is not the most efficient thing because SOAP/HTTP calls are just doggone slow when compared to MQ calls.
Just today, I attended an IIB 10 POT at IBM Hursley. One of the lab examples we did was just this, calling one service from another on the same broker.
I am sure that Vitor will remember it well.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 15, 2015 9:58 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| smdavies99 wrote: |
Just today, I attended an IIB 10 POT at IBM Hursley. One of the lab examples we did was just this, calling one service from another on the same broker.
I am sure that Vitor will remember it well. |
I remember happy days at Hursley in June.
Nice to hear they're reusing the labs.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| inMo |
| Posted: Wed Jul 15, 2015 10:26 am Post subject: Re: Broker calling itself |
|
|
Master
Joined: 27 Jun 2009
Posts: 216
Location: NY
|
| Vitor wrote: |
| If the "service" needs to be orchestrated it's a process and is better placed in Process Server or other such technologies. |
First, IMHO, your points are all reasonable. Second, forgive me for being perhaps a bit too literal, but orchestration itself is not a dirty word in WMB/IIB and by itself does not demand Process Server or the like. Many usage definitions call out orchestration as a core WMB/IIB capability. I think the manner and requirements of the proposed orchestration is the focal point. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 15, 2015 10:37 am Post subject: Re: Broker calling itself |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| inMo wrote: |
| Vitor wrote: |
| If the "service" needs to be orchestrated it's a process and is better placed in Process Server or other such technologies. |
First, IMHO, your points are all reasonable. Second, forgive me for being perhaps a bit too literal, but orchestration itself is not a dirty word in WMB/IIB and by itself does not demand Process Server or the like. Many usage definitions call out orchestration as a core WMB/IIB capability. I think the manner and requirements of the proposed orchestration is the focal point. |
I'll stand the correction. Would you accept that a "long running orchestration" is better off in WPS, but a transactional orchestration (such as a short SLA fan-in / fan-out) can sit inside WMB/IIB?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| inMo |
| Posted: Wed Jul 15, 2015 11:15 am Post subject: |
|
|
Master
Joined: 27 Jun 2009
Posts: 216
Location: NY
|
|
| Back to top |
|
|
| nelson |
| Posted: Thu Jul 23, 2015 6:04 am Post subject: |
|
|
Partisan
Joined: 02 Oct 2012
Posts: 313
|
Thanks all for the replies.
I know that technically it is not an issue to the broker to call itself...
| Vitor wrote: |
| What you shouldn't have inside broker is a process, especially a user driven or "long running" process. Broker is an integration engine and should not maintain state. Now again it's a design decision when a flow becomes a process, but in my view the fact that a flow has to call multiple services doesn't make it so. Broker certainly shouldn't be orchestrating in the way you describe, and if you're calling out to multiple services the design needs to account for that (probably using asynchronous techniques). If the "service" needs to be orchestrated it's a process and is better placed in Process Server or other such technologies. |
Very precise.
| smdavies99 wrote: |
A business level service would call one or more business services or technical service and these could call component services.
All implemented on the same broker as well! |
When a business level service could be seen as a process...(when it calls more business services?) ? I agree, calling technical/component services it's ok, but calling business level services... that's in fact my original question. 
Thanks all for your opinions. |
|
| Back to top |
|
|
| nelson |
| Posted: Thu Jul 23, 2015 6:14 am Post subject: |
|
|
Partisan
Joined: 02 Oct 2012
Posts: 313
|
Would you agree that processes with no human intervention could be orchestrated within the broker...? we are taking business decisions, but not storing any state...?  |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Jul 23, 2015 6:27 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| nelson wrote: |
| Would you agree that processes with no human intervention could be orchestrated within the broker...? we are taking business decisions, but not storing any state...? |
Depends on what you mean by orchestration.
Can a broker flow call multiple other services (including broker flows) using synchronous or asynchronous calls, and process the results from each of these as needed?
Yes.
Can it maintain a global transaction across all of these, without lots of user code?
No. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Jul 23, 2015 6:46 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| nelson wrote: |
| Would you agree that processes with no human intervention could be orchestrated within the broker...? we are taking business decisions, but not storing any state...? |
I would agree with this, inserting immediately the same caveat as my most worthy associate. If you have that number of processes, with no state stored, you're going to tie yourself in knots trying to roll back in the event of a failure. Your pain is compounded if any of the business processes that you're invoking cannot simply be rolled back and needs to be compensated; consider as an example an order process. You receive an order (let's assume on an MQ queue); you can validate the customer details, check his credit, check stock, approve the order (credit / balance against order value), call out to the web service that starts the invoice process and then start calling out to web services to physically provision his order. One of these throws an HTTP 500 at you and you can certainly roll the flow back and put the message back on the queue.
Even if you're doing the provisioning as an aggregation, one or more of the services has returned a 200 and is in the process of getting stock off the shelves. As part of the roll back, you have to identify which services worked and make a call to their "cancel order" web service. You also have to cancel the invoice. If it's a transitory error and your flow retry works in a short period, you then have a "produce invoice", "cancel invoice" and "produce invoice" all in your work stack of your invoicing system. One fairly trivial error in that code and you get two duplicate invoices and a "cancel what invoice?" error. Plus 1-n provisioning systems loading duplicate (and not charged for) items onto a truck.
A slightly contrived example perhaps, but intended to show that there's a point where, though technically feasible, the solution requires the product to be bent double to achieve it.
My personal experience (and I highlight that this is a personal viewpoint) is that all of these, long running, business driven message flows are great so long as every component works; the happy path tests out just great. It's when you try and deal with error situations that the wheels fall off. Even the example I gave above is perfectly feasible as long as all of the external calls work. It also assumes that the business process encapsulated by the flow has a rigid boundary that excludes the invoicing and provisioning process. If instead the business process requires that the provisioning be stopped if the invoicing fails (e.g. the customer is black listed for a previously unpaid invoice) then you need to write another convoluted flow to achieve a result that Process Server can achieve out of the box.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| nelson |
| Posted: Thu Jul 23, 2015 1:46 pm Post subject: |
|
|
Partisan
Joined: 02 Oct 2012
Posts: 313
|
| Vitor wrote: |
| nelson wrote: |
| Would you agree that processes with no human intervention could be orchestrated within the broker...? we are taking business decisions, but not storing any state...? |
I would agree with this, inserting immediately the same caveat as my most worthy associate. If you have that number of processes, with no state stored, you're going to tie yourself in knots trying to roll back in the event of a failure. Your pain is compounded if any of the business processes that you're invoking cannot simply be rolled back and needs to be compensated; consider as an example an order process. You receive an order (let's assume on an MQ queue); you can validate the customer details, check his credit, check stock, approve the order (credit / balance against order value), call out to the web service that starts the invoice process and then start calling out to web services to physically provision his order. One of these throws an HTTP 500 at you and you can certainly roll the flow back and put the message back on the queue.
Even if you're doing the provisioning as an aggregation, one or more of the services has returned a 200 and is in the process of getting stock off the shelves. As part of the roll back, you have to identify which services worked and make a call to their "cancel order" web service. You also have to cancel the invoice. If it's a transitory error and your flow retry works in a short period, you then have a "produce invoice", "cancel invoice" and "produce invoice" all in your work stack of your invoicing system. One fairly trivial error in that code and you get two duplicate invoices and a "cancel what invoice?" error. Plus 1-n provisioning systems loading duplicate (and not charged for) items onto a truck.
A slightly contrived example perhaps, but intended to show that there's a point where, though technically feasible, the solution requires the product to be bent double to achieve it.
My personal experience (and I highlight that this is a personal viewpoint) is that all of these, long running, business driven message flows are great so long as every component works; the happy path tests out just great. It's when you try and deal with error situations that the wheels fall off. Even the example I gave above is perfectly feasible as long as all of the external calls work. It also assumes that the business process encapsulated by the flow has a rigid boundary that excludes the invoicing and provisioning process. If instead the business process requires that the provisioning be stopped if the invoicing fails (e.g. the customer is black listed for a previously unpaid invoice) then you need to write another convoluted flow to achieve a result that Process Server can achieve out of the box. |
Sounds like a headache, and in fact it is .. Enough reason to start looking the idea to acquire some business process management tool...
Thanks all for your inputs. |
|
| Back to top |
|
|
| inMo |
| Posted: Fri Jul 24, 2015 11:43 am Post subject: |
|
|
Master
Joined: 27 Jun 2009
Posts: 216
Location: NY
|
| Or to pass on monolithic flow design in favor of a series of discreet units of work that avoid (minimize if you must) complex compensation scenarios. This requires elegant design where the integration team has strong influence and can cooperatively require applications be active participants in the solution. |
|
| Back to top |
|
|
|
|
