| Author |
Message
|
| gag_nm |
 Posted: Tue Mar 22, 2011 11:32 pm Post subject: Help in Design |
 |
|
Centurion
Joined: 16 Oct 2008
Posts: 102
|
Hi Friends,
i have some requirement as follows.
i have 4 different external-Vendor applications , MQ and HTTP connectivity is used for this vendors.
i should send request to 4 vendor applications concurrently. i should receive response back from this vendors and check each response, whether it is successfull or not.
my question, which nodes i need to use???
i was thinking may be Aggression nodes will help me.
or please suggest me which nodes i need to use?? |
|
| Back to top |
|
 |
| vmcgloin |
| Posted: Wed Mar 23, 2011 12:30 am Post subject: |
|
|
Knight
Joined: 04 Apr 2002
Posts: 560
Location: Scotland
|
What do you want to do with the responses? What if one or more applications does not respond in a timely way?
What other options are you considering/have you tried?
Aggregation nodes seem a sensible option (aggression nodes should be avoid  ) |
|
| Back to top |
|
|
| gag_nm |
| Posted: Wed Mar 23, 2011 12:49 am Post subject: |
|
|
Centurion
Joined: 16 Oct 2008
Posts: 102
|
If response message is successful, then next immediate request i will send it to respective vendor, who has sent me successful message.
In case if vendor doesn't respond in time,i will store it as Time out from respective vendor.
i tried using Aggression nodes, but it is sending request in sequence ,not in concurrent way.
how can i send messages in concurrently . |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Wed Mar 23, 2011 3:30 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
We consider your design requirement a complex mediation. This means that it is possible if one of the four downstream providers provides a negative response or timed out response, you may need to roll-back the previous transactions.
Complex mediations are inherently sticky. What I mean by this is, they are not easy to accomplish using only nodes. You have to have a more intelligent construct within your system to manage the state of the complex mediation and determine what action to do next along the way. Some people say this is a context manager, but I say it is more than managing context.
A common way to implement this mediation intelligence is through a Singleton, which runs in a separate thread in the background inside your DataFlowEngine Execution Group process.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Mar 23, 2011 4:15 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Please make sure I understand the question. I think there are many comments here that are not addressing the question that is being asked.
Is this the requirements?
You want to send the same request to 4 vendors.
You want to do this in parallel.
You want to save the state of the requests - vendor A was successful, vendor B was successful, vendor C timed out, vendor D timed out - for the next set of requests.
For the next set of requests, you want to send only to those vendors that were successful (A and B in the sample).
If a vendor times out, you do not need to do anything.
If these are the requirements then you need to understand that nothing that happens inside a single flow instance in Broker runs in parallel. The usual mechanism for making parallel requests is to use multiple flows. So what you would do is create a new flow that had MQinput-><nodes to make the request to a single vendor>->MQReply. Then this would be the thing that you aggregate - the response messages from all of these 4 flows.
This configuration would also let you understand what all 4 responses were in the context of the original message - particularly if you took some steps to add the original message to the aggregation. Then you could make the necessary compensating transactions if you really had a requirement to do so, without using a singleton or any other custom code - but it does not seem like you have a requirement for compensating transactions?
You could also write your own code in a JavaCompute or PHPCompute to create new threads and run the requests to the vendors in parallel that way - but this is significantly more advanced, and more complicated to maintain over time. |
|
| Back to top |
|
|
| gag_nm |
| Posted: Wed Mar 23, 2011 4:20 am Post subject: |
|
|
Centurion
Joined: 16 Oct 2008
Posts: 102
|
| Thanks Jeff for advice, i will try to do it using Java Compute node. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Mar 23, 2011 4:28 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
You really shouldn't use threads in a JavaCompute node unless you have extensive experience already with creating and running threads in Java.
And you really should use Broker standard methods - multiple flows and multiple instances - wherever possible.
This will ensure that the solution is maintainable and supportable. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Wed Mar 23, 2011 5:16 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
For transmitting the message to 4 recipients, one could use MQOutput with a 4x DestinationList. Or wire x4 to 4 separate MQOutputs or 4 separate subflows.
Transmitting the data is the easy part. How do you determine success/fail without an intelligence that can associate responses and evaluate the decision tree?
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Mar 23, 2011 6:01 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Logic to process the responses and evaluate the decision tree is handled in the Aggregation pattern, after the replies are or are not received.
Granted this requires business specific code, which is in a sense custom code - but it doesn't require building a complicated set of non-Broker native infrastructure.
In cases where one needs the original request message to make decisions, one usually forwards it to a flow that does an immediate reply with the same data, and then includes this in the whole Aggregation.
But on the other hand, this kind of complex orchestration is perhaps better suited for WPS than for Broker. Depends on the complexity of the situation and the comfort level. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Wed Mar 23, 2011 6:03 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| mqjeff wrote: |
| I think there are many comments here that are not addressing the question that is being asked. Then you could make the necessary compensating transactions if you really had a requirement to do so, without using a singleton or any other custom code - but it does not seem like you have a requirement for compensating transactions? You could also write your own code in a JavaCompute or PHPCompute to create new threads and run the requests to the vendors in parallel that way - but this is significantly more advanced, and more complicated to maintain over time. |
Packaging is an art form as well. I could not put the logic for the intelligence in a JCN. I would put the intelligence logic in a Singleton packaged as a JAR file stored in $MQSI_WORKPATH/shared-classes.
Having an intelligent mediation context management component would be a great feature for WMB v8.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Wed Mar 23, 2011 6:05 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| mqjeff wrote: |
| But on the other hand, this kind of complex orchestration is perhaps better suited for WPS than for Broker. Depends on the complexity of the situation and the comfort level. |
Very true. The Singleton solution is a step half-way between the Aggregation pattern and WPS. Singleton is lighter-weight than WPS but more intelligent than Aggregation.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Mar 23, 2011 6:23 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| lancelotlinc wrote: |
| Packaging is an art form as well. I could not put the logic for the intelligence in a JCN. I would put the intelligence logic in a Singleton packaged as a JAR file stored in $MQSI_WORKPATH/shared-classes. |
The problem with that is that you have to then ensure that all responses for a single aggregation are processed by the same EG.
And of course that one knows Java.
And you should really use a JavaClassLoader configurable service in v7 anyway, not shared-classes. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Wed Mar 23, 2011 6:28 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| mqjeff wrote: |
| lancelotlinc wrote: |
| Packaging is an art form as well. I could not put the logic for the intelligence in a JCN. I would put the intelligence logic in a Singleton packaged as a JAR file stored in $MQSI_WORKPATH/shared-classes. |
The problem with that is that you have to then ensure that all responses for a single aggregation are processed by the same EG.
And of course that one knows Java.
And you should really use a JavaClassLoader configurable service in v7 anyway, not shared-classes. |
Each instance of Singleton (one per execution group) communicates with the others. If a response is received by a different EG, it can be processed because the intelligence retrieves the needed info from the other EG's Singleton.
The shared-classes is recommended by the InfoCenter, and some Brokers here are still on WMB v6. Once all Brokers migrate to WMB v7, the plan is to have a custom classloader, which retrieves the JAR/classes from a WAS repository and can reload classes on-the-fly without resetting the Broker/EG or flows.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Mar 23, 2011 6:40 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
I have to side with Jeff on this one. It is easier to maintain and conceptually to run using the aggregation pattern and passing the original message to the aggregation as a way to maintain state.
A filter at the aggregation #2 can avoid passing the 2nd message to the timed out providers.
- Real easy to maintain.
- Does not require any advanced java knowledge (I have discovered that a lot of people know basic java, but very few do well with advanced java knowledge)
- should scale really nicely (multiple instances of the flow in the same e.g.).
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Wed Mar 23, 2011 6:52 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| fjb_saper wrote: |
| as a way to maintain state. |
My first post made a reference to the intelligence doing more than managing state/context. These are: (1) The upstream customers and downstream providers have an Internet-explorer-based tool to access a data repository to determine what happened with a particular request. Using MsgId, CorrelId, or ReferenceId, they can search a date range to find out the disposition of the request. (2) Business audit records are constructed and stored. (3) System events are constructed and logged. (4) Behavior of mediations are adjusted on-the-fly based on downstream QoS (ie. if one downstream is laggy, and another downstream is available to process requests, then DestinationLists are modified temporarily until the laggy downstream recovers. Lots more features, but these are the big ones.
| fjb_saper wrote: |
| a lot of people know basic java, but very few do well with advanced java knowledge. |
IMHO, the solution to this is more experience. Shying away from a challenge because you have not conquered it before is not a path to learning.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
|
|
