| Author |
Message
|
| sonalzade |
 Posted: Tue Oct 16, 2012 11:03 pm Post subject: Java compute node and message flow |
 |
|
Newbie
Joined: 16 Oct 2012
Posts: 5
|
Hi All
I am 2 years experienced in WMB but mostly worked with esql. Now I've to work with Java in Java compute node.
I've a requirement in which, there is a subflow getting the input mesg from the MQINPUT node of the main flow. The mesg is read as BLOB. I need to store the entire original input mesg permanently in subflow so that it can be used in future.
Currently, we are using a MQ to store this mesg. But, we need to remove this queue and find out a way to store it thru' code.
I am guessing we can do it using environment variable or collector node.
Pls guide me.
Thanks ! |
|
| Back to top |
|
 |
| Esa |
| Posted: Tue Oct 16, 2012 11:53 pm Post subject: Re: Java compute node and message flow |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| sonalzade wrote: |
I've a requirement in which, there is a subflow getting the input mesg from the MQINPUT node of the main flow. The mesg is read as BLOB. I need to store the entire original input mesg permanently in subflow so that it can be used in future.
|
Please be a little more specific with the requirements. Do you need to store all messages permanently or just one - perhaps some sort of a configuration message?
In the case of a configuration message there may be better alternatives, for example user defined configurable services.
If you need to store a message permanently but not in a queue, you can store it in a file or in a database, for example. |
|
| Back to top |
|
|
| sonalzade |
| Posted: Wed Oct 17, 2012 12:30 am Post subject: |
|
|
Newbie
Joined: 16 Oct 2012
Posts: 5
|
Thanks for the reply...
By "message" I mean to say the input message coming from the consumer. With complete assembly... MQMD+Environment+Payload.
The input mesg enters the subflow, we need to retain a copy of it and the mesg is then propagated to the further nodes (wtx node, compute nodes, SOAP nodes) and then just before coming out of that subflow, we again need the original copy of message (the same MQMD+Env+Payload) that entered into the subflow.
I hope I am able to convey.
As mentioned, storing in a file or a database.
I would like to tell, that the number of mesgs that this subflow will get may exceed some thousands of records... so which way should be the most efficient one ? |
|
| Back to top |
|
|
| Gemz |
| Posted: Wed Oct 17, 2012 1:04 am Post subject: |
|
|
Centurion
Joined: 14 Jan 2008
Posts: 124
|
What about flow order node? (This is for MQMD & Payload)
_________________
GemZ
"An expert is one who knows more and more about less and less until he knows absolutely everything about nothing...." |
|
| Back to top |
|
|
| Esa |
| Posted: Wed Oct 17, 2012 1:09 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| sonalzade wrote: |
The input mesg enters the subflow, we need to retain a copy of it and the mesg is then propagated to the further nodes (wtx node, compute nodes, SOAP nodes) and then just before coming out of that subflow, we again need the original copy of message (the same MQMD+Env+Payload) that entered into the subflow.
|
Does not sound very permanent to me, you will just store the message during part of the thread execution time.
The MQInput node already keeps the copy for you.
| sonalzade wrote: |
| I am guessing we can do it using environment variable or collector node. |
Your guess is good.
My guess is that you have to combine the message produced by the subflow with the original message or do some other processing where you need to access both messages simultaneously. If you don't need the messages simultaneously, you don't need to store the input message... |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Oct 17, 2012 3:51 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Gemz wrote: |
| What about flow order node? (This is for MQMD & Payload) |
This does exactly what is being asked for. The message goes down the first terminal. When that terminal completes, it goes down the second terminal exactly the same as it went down the first one. Only Environment is different. |
|
| Back to top |
|
|
| Esa |
| Posted: Wed Oct 17, 2012 4:15 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| mqjeff wrote: |
| Only Environment is different. |
Only LocalEnvironment is different so that any changes to it you made after terminal 1 won't be there after terminal 2, because FlowOrder copies its InputLocalEnvironment to OuputLocalEnvironment for both terminals separately.
So if you decide to store the input message (or the output message produced by the subflow, whichever is smaller), store it in Environment. Or use the Collector node, but that will use a system queue internally which may be against your requirement not to use queues for the storage.
If you use a Collector node, you don't in fact need the FlowOrder node.
The above mentioned applies to the case where you need to combine the transformed message with the original. If you don't need to do it, just a FlowOrder node should be enough. And you wouldn't need even that if your transformation flow did not contain SOAP nodes, which do not support transactions.
Last edited by Esa on Wed Oct 17, 2012 4:22 am; edited 1 time in total |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Oct 17, 2012 4:21 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Esa wrote: |
| mqjeff wrote: |
| Only Environment is different. |
Only LocalEnvironment is different so that any changes to it you made after terminal 1 won't be there after terminal 2, |
That's not "different". That's "Local Environment is the *same* after each terminal.
There is only one Environment so all changes made to it are made to it everywhere, so when you propagate to terminal 2 of FlowOrder, all changes made to Environment along terminal 1 are still changed. All changes made to Local Environment along terminal 1 are NOT still changed.
| Esa wrote: |
| If you use a Collector node, you don't in fact need the FlowOrder node. |
I would put it exactly the other way - if you use Flow Order, you don't in fact need the Collector node.
And Collector is heaver and more complicated and uses additional units of work, which may not be what is required (in fact, likely isn't for an error handling subflow). |
|
| Back to top |
|
|
| Esa |
| Posted: Wed Oct 17, 2012 4:43 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| mqjeff wrote: |
That's not "different". That's "Local Environment is the *same* after each terminal.
There is only one Environment so all changes made to it are made to it everywhere, so when you propagate to terminal 2 of FlowOrder, all changes made to Environment along terminal 1 are still changed. All changes made to Local Environment along terminal 1 are NOT still changed.
|
Yes. We mean the same thing but we are looking at it from opposite directions.
Which nodes the OP should use depends on what she needs to do with the messages. And she hasn't said anything else that she needs the original message at the end of the subflow.
If you just need to store the original message in Environment, there is no specific need for just FlowOrder, a Filter node can do that, and a Compute node in correct computing mode and many others... |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Oct 17, 2012 4:49 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Esa wrote: |
| Yes. We mean the same thing but we are looking at it from opposite directions. |
Yes.
| Esa wrote: |
| Which nodes the OP should use depends on what she needs to do with the messages. And she hasn't said anything else that she needs the original message at the end of the subflow. |
Right. Which is why the FlowOrder is the best option, in that it is the absolute least work to accomplish the goal. Wire it in between the Input node of the subflow and the rest of the subflow, and wire the second terminal of the Flow Order to the Output node of the subflow. *done*.
| Esa wrote: |
| If you just need to store the original message in Environment, there is no specific need for just FlowOrder, a Filter node can do that, and a Compute node in correct computing mode and many others... |
Right. But it may be, again, simpler and require less work, to use flow order because then you don't need to copy the message to Environment and then copy it back.
 |
|
| Back to top |
|
|
| sonalzade |
| Posted: Fri Nov 23, 2012 12:39 am Post subject: Removing temporary queue |
|
|
Newbie
Joined: 16 Oct 2012
Posts: 5
|
Hi,
Thanks a lot, everybody for your response!
Sorry for delay in response as i was on vacation and working on other urgent defects.
Sorry for not being clear about the requirements. Let me elaborate which should help understanding the problem:
In the system, there are different subflows which communicates to different systems. We get input message from queue in the Main Flow and the same is passed to a Routing Sub Flow. Here we can not replace the Routing Sub Flow with the Route node as to determine routing we need to call few services etc. The Routing Sub Flow based on processing output determines to which subflow to route the incoming message. The processing message is not required once Routing destination is determined.
Currently it implemented as follow: message enters Routing Sub Flow which first stores the incoming message to one temp queue. It then processes transaction, determines the Routing destination and then picks up the incoming message from the temp queue and forwards to the appropriate destination sub flow.
Now the defect is raised mentioning that the temp queue is redundant and we should store the incoming message in env variable and look it up later and forward to destination sub flow.
We tried the solution by storing incoming message in env variable and later forwarding to sub flow. It is successful for one sub flow but in the other where a collector node is used, it's failing. The collector node in the destination sub flow is throwing below exception: BIPmsgs: 6105, Invalid buffer parameters and also mentioning about HTTPInputHeader and WSINPHDR. I am not able to attach the exception in a file, hence, pls find the exception text below. The flow then stucks going in an infinite loop, getting stucked in Collector node.
ExceptionList
RecoverableException
File:CHARACTER:/build/S700_P/src/DataFlowEngine/ImbDataFlowNode.cpp
Line:INTEGER:1083
Function:CHARACTER:ImbDataFlowNode::createExceptionList
Type:CHARACTER:ComIbmCollectorNode
Name:CHARACTER:EB_270_ROUTE_MAIN_FLOW#FCMComposite_1_20.EB_270_MAPTO_271FACADEREQUEST_SUB_FLOW#FCMComposite_1_1
Label:CHARACTER:EB_270_ROUTE_MAIN_FLOW.EB_270_MAPTO_271FACADEREQUEST_SUB_FLOW.Collector
Catalog:CHARACTER:BIPmsgs
Severity:INTEGER:3
Number:INTEGER:2230
Text:CHARACTER:Node throwing exception
ParserException
File:CHARACTER:/build/S700_P/src/DataFlowEngine/ImbRootParser.cpp
Line:INTEGER:816
Function:CHARACTER:ImbRootParser::parseNextItem
Type:CHARACTER:ComIbmCollectorNode
Name:CHARACTER:EB_270_ROUTE_MAIN_FLOW#FCMComposite_1_20.EB_270_MAPTO_271FACADEREQUEST_SUB_FLOW#FCMComposite_1_1
Label:CHARACTER:EB_270_ROUTE_MAIN_FLOW.EB_270_MAPTO_271FACADEREQUEST_SUB_FLOW.Collector
Catalog:CHARACTER:BIPmsgs
Severity:INTEGER:2
Number:INTEGER:5902
Text:CHARACTER:Exception whilst parsing
Insert
Type:INTEGER:5
Text:CHARACTER:Root
Insert
Type:INTEGER:5
Text:CHARACTER:HTTPInputHeader
Insert
Type:INTEGER:5
Text:CHARACTER:Null Buffer
ParserException
File:CHARACTER:/build/S700_P/src/WebServices/WSLibrary/ImbWSHeaderParsers.cpp
Line:INTEGER:249
Function:CHARACTER:ImbWSInputHeaderParser::refreshElementsFromBitStream
Type:CHARACTER:ComIbmCollectorNode
Name:CHARACTER:EB_270_ROUTE_MAIN_FLOW#FCMComposite_1_20.EB_270_MAPTO_271FACADEREQUEST_SUB_FLOW#FCMComposite_1_1
Label:CHARACTER:EB_270_ROUTE_MAIN_FLOW.EB_270_MAPTO_271FACADEREQUEST_SUB_FLOW.Collector
Catalog:CHARACTER:BIPmsgs
Severity:INTEGER:3
Number:INTEGER:6105
Text:CHARACTER:Invalid buffer parameters
Insert
Type:INTEGER:5
Text:CHARACTER:HTTPInputHeader
Insert
Type:INTEGER:5
Text:CHARACTER:WSINPHDR
We googled for the exception and found only one relevant post:
http://www.mqseries.net/phpBB/viewtopic.php?p=241606&sid=2d90d4da8ea58db1f219eafca844d53e
The exception in the above post is Null Buffer but we are getting Invalid Buffer parameters but initially we got this exception as well). The reply mentions that this problem occurs in Linux system, the system which we are using. We are already using the java compute node to copy the message headers and the incoming message to BLOB/BLOB. But we are still getting above mentioned exception.
We even tried to propagate the exact same incoming message. We stored incoming message in map against correlationId and then upon determining Routing destination, look up the same incoming MbMessage from the map against correlationId and forward to sub flow. But the issue persists.
Here are the Broker details:
OS Name: Linux
OS Version: #1 SMP 2010-05-20 11:14:20 +0200
OS Architechture: s390x
WMB Version: 7.0.0.5
Build Level: S700-FP05
Pl advise. Looking forward to getting your reply.
Thanks n regards,
Sonal Zade |
|
| Back to top |
|
|
|
|
