| Author |
Message
|
| TonyD |
 Posted: Wed Dec 19, 2007 2:16 pm Post subject: Aggregate Nodes Question |
 |
|
Knight
Joined: 15 May 2001
Posts: 540
Location: New Zealand
|
I have a flow that sends an MQ request message to an external app and uses an MQGet loop to receive up to 700 replies with the same CorrelId, terminated with an 'End Of Replies' message.
It has been suggested that I should be using aggregate nodes rather than MQGet, for thread blocking reasons.
Its been a while since I last used aggregate nodes but a quick test showed that reply messages 2-n were rejected because there were no '..corresponding request messages', which is what I expected. Even if I used logical groups I think the result would be the same; it would just mean that the first reply would not be read until the last reply had arrived on the queue
Is there a technique that would allow me to use aggregate nodes for this scenario? I do not know in advance how many reply messages will be returned, hence the need for the 'End OF Replies' marker. |
|
| Back to top |
|
 |
| EddieA |
| Posted: Wed Dec 19, 2007 3:34 pm Post subject: Re: Aggregate Nodes Question |
|
|
Jedi
Joined: 28 Jun 2001
Posts: 2453
Location: Los Angeles
|
| TonyD wrote: |
| Is there a technique that would allow me to use aggregate nodes for this scenario? |
No. Aggregation assumes 1 Reply per Request.
Cheers,
_________________
Eddie Atherton
IBM Certified Solution Developer - WebSphere Message Broker V6.1
IBM Certified Solution Developer - WebSphere Message Broker V7.0 |
|
| Back to top |
|
|
| sunny_30 |
| Posted: Wed Dec 19, 2007 6:45 pm Post subject: |
|
|
Master
Joined: 03 Oct 2005
Posts: 258
|
Tony,
Eddie is right that MB aggregation assumes 1 reply for 1 request.
But, how about you making the MB believe that you have sent 700 requests, instead you just send one. The aggregation then waits for 700 replies back.
Try below (for K=2 to 700):-
This code needs to be added in the node after MQOutput (where one request is already sent) & before the aggregaterequest node.
| Code: |
SET OutputLocalEnvironment.WrittenDestination.MQ.DestinationData[K].queueName = 'XXXXXXXXXXXX';
SET OutputLocalEnvironment.WrittenDestination.MQ.DestinationData[K].queueManagerName = 'XXXXXXXXXXXX';
SET OutputLocalEnvironment.WrittenDestination.MQ.DestinationData[K].replyIdentifier = UUIDASBLOB || X'0000000000000000';
SET OutputLocalEnvironment.WrittenDestination.MQ.DestinationData[K].msgId = UUIDASBLOB || X'0000000000000000';
SET OutputLocalEnvironment.WrittenDestination.MQ.DestinationData[K].correlId = X'000000000000000000000000000000000000000000000000'; |
The above code will work. But will not guarantee that the responses received are for the same request made.
Also try below, Im not sure this works, but if it works, you can guarantee that 700 responses received are for the same request made i.e not for the request from the other instance of the flow:
| Code: |
SET OutputLocalEnvironment.WrittenDestination.MQ.DestinationData[K].queueName = 'XXXXXXXXXXXX';
SET OutputLocalEnvironment.WrittenDestination.MQ.DestinationData[K].queueManagerName = 'XXXXXXXXXXXX';
DECLARE BLOB MsgIdOfFirstRequest = InputLocalEnvironment.WrittenDestination.MQ.DestinationData[1].msgId;
SET OutputLocalEnvironment.WrittenDestination.MQ.DestinationData[K].replyIdentifier = MsgIdOfFirstRequest;
SET OutputLocalEnvironment.WrittenDestination.MQ.DestinationData[K].msgId = MsgIdOfFirstRequest;
SET OutputLocalEnvironment.WrittenDestination.MQ.DestinationData[K].correlId = MsgIdOfFirstRequest; |
Note that for Aggregation MB matches the CorrelId of the response to the MsgId of the request made.
Let me know what you find after testing...
-Sunny. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Dec 19, 2007 8:16 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
I believe this would be one for V6.1 which has a collection node to do exactly what you're looking for? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| TonyD |
| Posted: Thu Dec 20, 2007 6:21 pm Post subject: |
|
|
Knight
Joined: 15 May 2001
Posts: 540
Location: New Zealand
|
Thanks for the comments.
| Quote: |
But, how about you making the MB believe that you have sent 700 requests, instead you just send one. The aggregation then waits for 700 replies back.
|
I wondered about something along those lines. However I do not know in advance how many reply messages will be returned. If I set a maximum the Aggregate Reply will keep waiting for messages that will never arrive and will then timeout.
| Quote: |
I believe this would be one for V6.1 which has a collection node to do exactly what you're looking for?
|
I do not yet have 6.1, but have read the Collection node documentation. The format of my reply messages (not XML) makes me wonder if I would be able to suitably configure an 'event handler' to detect the end of the collection.
In fact the MQGet loop works perfectly satisfactorily for any number of reply messages up to the 1000 maximum that I have set. The consultant's comment was:
| Quote: |
Some Broker flows are using the MQGet node for long-running requests. The MQGet node is intended to be used for short-running requests such as a SOAP/JMS Web Service call. For long running requests, use of the Aggregate nodes is recommended. The advantage of message aggregation is that the separate WMB flows for the requests and replies are non-blocking and can handle a high throughput with a single thread. With the MQGet node, the message flow thread is blocked until a reply is received or the MQGet node times out.
|
As we are not seeing performance problems I think that I will leave things the way they are for the time being! |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Dec 20, 2007 8:04 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Is it possible to process the responses individually?
Or do you need to assemble all of them before you can process any of them?
If you can process them individually, you'll get vastly better performance out of using an MQInput rather than an MQGet.
On the third hand, you should evaluate the risk of performance/memory issues with your current setup, against the cost of making changes.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| Cliff |
| Posted: Fri Dec 21, 2007 2:10 am Post subject: |
|
|
Centurion
Joined: 27 Jun 2001
Posts: 145
Location: Wiltshire
|
Just to add another design consideration, WMQ is asynchronous and cannot guarantee message sequence unless certain strict criteria are met so do you cater for your 'end of replies' message not actually being the last message? Believe me, it WILL happen at some time.
Aggregation can help here, but you would have to do something like sending one 'real' request message and 699 dummy ones, while the replying app would have to send (say) 650 'real' replies and 50 dummy ones.
Just a thought ... |
|
| Back to top |
|
|
|
|
