Author |
Message
|
ddg |
Posted: Wed Aug 29, 2007 10:00 am Post subject: Aggregating Original Request message to a Reply Problem |
|
|
Newbie
Joined: 29 Aug 2007 Posts: 6
|
I am in WMB6 trying to do a Fan-out/Fan-In (Aggregation) flow setup. This is my first time creating
a aggregation flow setup so bear with me.
Error:
I always get this in User Trace for Fan-In:
2007-08-29 10:49:10.324821 5944 UserTrace BIP4412I: Corresponding reques
t record not found for the reply message.
An AggregateReply node has received a mes
sage at its 'in' terminal. No corresponding record of a request message being se
nt could be found. See subsequent messages to determine how this situation has b
een handled.
It is possible that extraneous messages a
re arriving at the AggregateReply node 'in' terminal. Check your flow to ensure
that the only messages arriving here are replies to request messages previously
sent out and passed through an AggregateRequest node. It is possible that this m
essage is a valid reply but part of an aggregation which previously timed out. I
t is possible that this is a reply to a message which has not yet been recorded
by an AggregateRequest node. This can happen if request messages are sent outsid
e of transactional control. Adjust your transaction settings to ensure that mess
ages are sent under transactional control.
2007-08-29 10:49:10.326069 5944 UserTrace BIP4413I: Storing unknown mess
age persistently for subsequent processing.
The message has been stored persistently
by the AggregateReply node. If it is subsequently discovered to be a valid reply
in an aggregation then it will be processed as such at that time. If a number o
f seconds passes as specified by the 'unknown timeout' attribute, then the messa
ge will be propagated to the 'unknown' terminal.
No user action required.
I have an XML message coming in to a
Fan-out flow:
- InputQ to AggControlNode,
- AggControlNode splits to Compute1 and Compute2,
- Each Compute* goes to MQOutputNodes (mqOUT1 and mqOUT2(same queue as fan-in InputQueue later below))
- and then to 2 respective AggRequestNodes
Compute 1:
Set OutputRoot = InputRoot;
SET OutputLocalEnvironment.ComIbmAggregateControlNode =
InputLocalEnvironment.ComIbmAggregateControlNode;
Set OutputRoot.BLOB.BLOB to <some XMLNS field's value>;
SET OutputRoot.XMLNS = NULL;
SET OutputRoot.MQMD.CorrelId = InputRoot.MQMD.MsgId;
Compute 2:
Set OutputRoot = InputRoot;
SET OutputLocalEnvironment.ComIbmAggregateControlNode =
InputLocalEnvironment.ComIbmAggregateControlNode;
SET OutputRoot.MQMD.CorrelId = InputRoot.MQMD.MsgId;
Now, mqOUT1 mentioned above is the same as the inputQ to another flow for processing.
This processing flow has:
inputQ(mqOUT1) ==> *OutputQ
(*OutputQ being the same as input queue to the Fan-In InputQueue later below)
The reason for this is I am trying to mimick/simulate another system getting a blob then doing something to it
and then sending it back again as blob.
Fan-In:
InputQueue ==> AggReplyNode
AggReplyNode's Unknown Terminal ==> Compute_1
AggReplyNode's Out Terminal ==> Compute_2
AggReplyNode's TimeOut Terminal ==> Compute_3
Compute_1, _2 and _3 format the message to send particular folders after aggregation to go to queues for logging.
In addition:
- I might have some Trace Nodes in different places just to keep track of ${Root} and/or ${LocalEnvironment}.
- AggCOntrolNode has Setting of 300 secs for now
- AggReplyNode, which has the same Aggregate Name as AggControlNode, has unknown messaeg Timeout of 5 for now.
Any help/suggestion is greatly appreciated. Sorry for the very long post. I wanted to give as much info as I can. |
|
Back to top |
|
 |
vk |
Posted: Wed Aug 29, 2007 10:36 am Post subject: |
|
|
Partisan
Joined: 20 Sep 2005 Posts: 302 Location: Houston
|
Hi ddg,
In the message flows picking up from the queues mqOut1 and mqOut2, are you copying the MsgId in the MQMD header to the CorrelId before putting the message into the Reply queue (Input queue for the Fan-In flow)?
The AggregateReply node will check the CorrelId of each message it receives and then compare it with the MsgId of all request messages which are present in the queue SYSTEM.BROKER.AGGR.REQUEST. If a matching request is found, it is treated as a valid reply and will be places in the SYSTEM.BROKER.AGGR.REPLY queue. Else, it will be treated as an unknown message and sent to the SYSTEM.BROKER.AGGR.UNKNOWN queue.
Also, in the Fan-Out flow mke sure that you select the property 'New Message ID' in the MQOutput nodes. This would ensure that each request message has a unique MsgId.
Regards,
VK. |
|
Back to top |
|
 |
ddg |
Posted: Wed Aug 29, 2007 11:05 am Post subject: |
|
|
Newbie
Joined: 29 Aug 2007 Posts: 6
|
Hi vk,
First of all thanks for the reply.
So....
When you said, "In the message flows picking up from the queues mqOut1 and mqOut2, are you copying the MsgId in the MQMD header to the CorrelId before putting the message into the Reply queue (Input queue for the Fan-In flow)? "
I am not sure if I understood the question completely but here is an attempt to answer/clarify:
I have the copying of the MsgID to CorrelID of the MQMD in the 2 compute nodes in the fan-out flow ("Compute 1" and "Compute 2") BEFORE they get sent to mqOUT1 and mqOUT2 and then ultimately to their own respetive Request nodes. I hope this answers your question/clarification.
Also, I will try checking "New Message ID" in the MQOutput nodes in the fan-out flow and re-test. |
|
Back to top |
|
 |
vk |
Posted: Wed Aug 29, 2007 12:03 pm Post subject: |
|
|
Partisan
Joined: 20 Sep 2005 Posts: 302 Location: Houston
|
Hi ddg,
Quote: |
I have the copying of the MsgID to CorrelID of the MQMD in the 2 compute nodes in the fan-out flow ("Compute 1" and "Compute 2") BEFORE they get sent to mqOUT1 and mqOUT2 and then ultimately to their own respetive Request nodes. |
This is not correct. This will result in 2 messages having the same correlation ID.
Quote: |
Now, mqOUT1 mentioned above is the same as the inputQ to another flow for processing.
This processing flow has:
inputQ(mqOUT1) ==> *OutputQ
(*OutputQ being the same as input queue to the Fan-In InputQueue later below) |
I was referring to the above quote in your post. Copy MsgId of each request to the CorrelId in these flows which pick up messages from mqOUT1 and mqOUT2 queues.
Fan-Out Flow
Creates 2 requests and sends them to mqOUT1 and mqOUT2. In the output nodes in this flow, select 'New Message ID' property so that each request has a new message ID than the original messages which was picked up by the Fan-Out flow. The new message ID will be stored in the SYSTEM.BROKER.AGGR.REQUEST when the message passes through the AggregateRequest node after mqOUT1 and mqOUT2 nodes
Intermediate Flows
These are the flows which processes the fanned out requests and constructs the reply. In these flows copy MsgId of input message to CorrelId of output message before giving the output message to Fan-In flow
Fan-In Flow
Pick up replies from the InputQueue and perform aggregtion.
Hope this helps.
Regards,
VK. |
|
Back to top |
|
 |
ddg |
Posted: Wed Aug 29, 2007 12:21 pm Post subject: |
|
|
Newbie
Joined: 29 Aug 2007 Posts: 6
|
Thanks again.
I understand now. I actually can do that with the intermediate flow that picks up from mqOUT1.
However, mqOUT2 is actually the input queue to the Fan-In flow. I am sending one message to mqOUT1 for further processing (I understand this is where the MsgID to CorrelID copying should be) then I ALSO send the original message from fan-out straight to the Fan-IN to be aggregated with the "further processed one". So, send one out and send the other straight to fan-in for aggregation with a reply.
I can change the Intermediate Flow for mqOUT1 and I can copy new MsgID in there to CorrelID but how about the other leg that goes straight to the fan-in? |
|
Back to top |
|
 |
jefflowrey |
Posted: Wed Aug 29, 2007 12:37 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002 Posts: 19981
|
You can't send the second straight to fan-in.
You need to send all messages that will show up at fan-in through an AggregateRequest. And then the messages that show up at fan-in have to be reply messages to the messages that go through AggregateRequest _________________ I am *not* the model of the modern major general. |
|
Back to top |
|
 |
vk |
Posted: Wed Aug 29, 2007 12:47 pm Post subject: |
|
|
Partisan
Joined: 20 Sep 2005 Posts: 302 Location: Houston
|
Quote: |
However, mqOUT2 is actually the input queue to the Fan-In flow |
Aggregation is not designed to work that way. Fan out flow should generate requests. Each request should be processed by intermediate flows and send replies to Fan in flow.
Can you provide more details on the overall design requirement? What kind of additional processing is the intermediate flow doing in your case? It would help to see if you really need aggregation here.
Anyways for your current design, I can think of only 1 solution - Instead of giving mqOUT2 directly to the Fan In flow, give it to another intermediate flow and then to the Fan In flow as is done for mqOUT1.
Regards,
VK. |
|
Back to top |
|
 |
ddg |
Posted: Wed Aug 29, 2007 12:53 pm Post subject: |
|
|
Newbie
Joined: 29 Aug 2007 Posts: 6
|
Quote: |
You can't send the second straight to fan-in... You need to send all messages that will show up at fan-in through an AggregateRequest |
when I said "straight to fan-in...", I am sorry I misinformed. I meant it does not have an intermediate flow. I do send it through an AggregateRequest node though.
So....
second leg:
Agg control node to mqOUT2("new Message Id" checked and "Request" checked) to AggRequest node to Fan-In.
first leg:
(as mentioned in last post goes)
Agg control node to mqOUT1 ("new Message Id" checked and "Request" checked) then to AggregateRequest node to Intermediate flow then to Fan-In.
Now my revised and/or additional question is, given that the second leg goes straight to the fan-in through a Aggregate Request Node and is tagged as "Request" (per mqOUT2 properties), how do I make this part of the aggregation? |
|
Back to top |
|
 |
jefflowrey |
Posted: Wed Aug 29, 2007 12:54 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002 Posts: 19981
|
Send it to an MQReply node rather than an MQOutput node. _________________ I am *not* the model of the modern major general. |
|
Back to top |
|
 |
ddg |
Posted: Wed Aug 29, 2007 1:30 pm Post subject: |
|
|
Newbie
Joined: 29 Aug 2007 Posts: 6
|
Let me make sure I understand.
Fan Out Flow:
input queue to:
first leg:
Agg Control
to
some logic in a compute node
to
mqOUT1 ("New Message Id" checked and "Request" checked)
to
Agg Request Node
(then onwards for further processing for reply in an intermediate flow then to Fan-In)
second leg:
Agg Control
to
MQReply Node
to
Agg Request Node
(then onwards to, not to an intermediate flow, but to the fan-in for aggregation with first leg's reply)
Is this what you mean, jefflowrey? Do I therefore have to have a compute node before the MQReply node to set the MQMD.ReplyToQ? Also, won't the MQMD.CorrelId come out of this leg as X'000000000000000000000000000000000000000000000000' ? |
|
Back to top |
|
 |
jefflowrey |
Posted: Wed Aug 29, 2007 1:38 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002 Posts: 19981
|
You need to go AggRequest->Compute->MQReply.
Every message that shows up in the Fan-In flow must be a reply message.
The easiest way to build a reply message is to use an MQReply node. _________________ I am *not* the model of the modern major general. |
|
Back to top |
|
 |
ddg |
Posted: Wed Aug 29, 2007 2:10 pm Post subject: |
|
|
Newbie
Joined: 29 Aug 2007 Posts: 6
|
THANK YOU very much, jefflowrey and vk for your help and your patience. That made the 2 messages arriving at the fan-in's input queue match ID's.
Thanks again. |
|
Back to top |
|
 |
|