| Author |
Message
|
| ata_nitjsr |
 Posted: Thu Nov 18, 2010 6:55 am Post subject: File Output Node Query |
 |
|
Acolyte
Joined: 08 Apr 2007
Posts: 56
|
Need a help:-
In a Queue there are 50 message of 3 different type (Eg a,b,c). My requirement is to create 3 different file (a.txt, b.txt, c.txt).
a.txt will have all message of a type, b.txt will have all message of b type, c.txt will have all message of c type.
I have connected "Finish File" terminal in file output node.
my problem is the type of message is last one (say c ) is processing successfully. Rest of the message is lying in mqsitransit (say a & b).
How to process all the type successfully?  |
|
| Back to top |
|
 |
| ata_nitjsr |
| Posted: Thu Nov 18, 2010 7:12 am Post subject: My flow |
|
|
Acolyte
Joined: 08 Apr 2007
Posts: 56
|
My flow has
MQInput -> MQGet (out terminal)-> Compute node -> (In)Fileoutput
--------------MQGet (No msg terminal)-> Compute node -> (End file)Fileoutput
http://b.imagehost.org/view/0838/a
---------------------------------------------------------
MQ get property set with Browse mode
---------------------------------------------------------
Both the compute node has same codeing : -
CREATE COMPUTE MODULE XML2File_Flow_Compute
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
CALL CopyEntireMessage();
DECLARE RO,FileName,CurDate CHARACTER;
SET RO = InputRoot.XMLNSC.RO;
--SET OutputLocalEnvironment.Destination.File.Directory = 'E:\File\' || RO;
SET OutputLocalEnvironment.Destination.File.Directory = 'E:\File';
SET CurDate = CURRENT_DATE ;
SET OutputLocalEnvironment.Destination.File.Name= RO || '_' || CurDate || '.txt';
END;
END MODULE;[img][/img]
Last edited by ata_nitjsr on Thu Nov 18, 2010 8:52 am; edited 5 times in total |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Nov 18, 2010 7:17 am Post subject: Re: My flow |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20767
Location: LI,NY
|
| ata_nitjsr wrote: |
My flow has
[img]C:\Documents and Settings\347636\Desktop[/img] |
If you want the image to show, you first need to upload it to some image server and post the URL between the image tags...
Your computer desktop is not accessible from this board.... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Nov 18, 2010 7:24 am Post subject: Re: File Output Node Query |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| ata_nitjsr wrote: |
| How to process all the type successfully? |
How have you configured the node/flow to produce the 3 file types; i.e. 3 separate files wth 3 message types going to each based on content?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| ata_nitjsr |
| Posted: Thu Nov 18, 2010 7:34 am Post subject: Re: File Output Node Query |
|
|
Acolyte
Joined: 08 Apr 2007
Posts: 56
|
| Vitor wrote: |
How have you configured the node/flow to produce the 3 file types; i.e. 3 separate files wth 3 message types going to each based on content? |
I have done File output node configuration in Record Def as "Record is Delimited Data" |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Nov 18, 2010 7:52 am Post subject: Re: File Output Node Query |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| ata_nitjsr wrote: |
| Vitor wrote: |
How have you configured the node/flow to produce the 3 file types; i.e. 3 separate files wth 3 message types going to each based on content? |
I have done File output node configuration in Record Def as "Record is Delimited Data" |
That's nice, and not what I asked, but you've edited your post to include code so that's that.
On an unrelated note, MQGet with browse is a bad idea. MQGet node in a flow with browse is worse.
If we assume (as you've not said) that RO contains the a, b, or c for your file you're generating 3 files from a single FileOutput node, then instructing the node to "Finish File", then acting surprised when it only finishes the one it had open for the last read? Am I right?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| ata_nitjsr |
| Posted: Thu Nov 18, 2010 9:00 am Post subject: Re: File Output Node Query |
|
|
Acolyte
Joined: 08 Apr 2007
Posts: 56
|
| Vitor wrote: |
If we assume (as you've not said) that RO contains the a, b, or c for your file you're generating 3 files from a single FileOutput node, then instructing the node to "Finish File", then acting surprised when it only finishes the one it had open for the last read? Am I right? |
yes RO will have a,b,c. all your understanding is perfect 
only last file is connected to the "Finish File" so it has end of file(EOF) so it has moved out of mqsitransit rest two file is in mqsitransit only. i need to have EOF in other two. can you propose me a good design?
my flow is in http://b.imagehost.org/view/0838/a
Last edited by ata_nitjsr on Thu Nov 18, 2010 9:07 am; edited 1 time in total |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Nov 18, 2010 9:06 am Post subject: Re: File Output Node Query |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| ata_nitjsr wrote: |
| can you propose me a good design? |
Well 3 File Output nodes would seem to be an obvious place to start. It also wouldn't require major changes to your compute node.
I repeat my objections to your use of MQGet, and your use of browse. Put some thought into redesigning that. There's a chance it'll bite you if message volumes rise.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| ata_nitjsr |
| Posted: Thu Nov 18, 2010 9:14 am Post subject: Re: File Output Node Query |
|
|
Acolyte
Joined: 08 Apr 2007
Posts: 56
|
| Vitor wrote: |
Well 3 File Output nodes would seem to be an obvious place to start. It also wouldn't require major changes to your compute node. |
3 file is an eg. It can have n different type of message. So i cant put that many file output node. i should be using 1 file output node & dynamically create the file
| Vitor wrote: |
I repeat my objections to your use of MQGet, and your use of browse. Put some thought into redesigning that. There's a chance it'll bite you if message volumes rise. |
I need to re-think or re-design the flow. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Nov 18, 2010 9:20 am Post subject: Re: File Output Node Query |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| ata_nitjsr wrote: |
| I need to re-think or re-design the flow. |
I think so.
Two thoughts for you to ponder:
1) Browsing messages is bad. Looping round a flow browsing messages is worse.
2) A Collector node could assemble all the different record types for you based on the value of RO
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| ata_nitjsr |
| Posted: Thu Nov 18, 2010 9:26 am Post subject: Re: File Output Node Query |
|
|
Acolyte
Joined: 08 Apr 2007
Posts: 56
|
| Vitor wrote: |
2) A Collector node could assemble all the different record types for you based on the value of RO |
I will try with collector node. remove mq Browsing |
|
| Back to top |
|
|
| joebuckeye |
| Posted: Thu Nov 18, 2010 10:23 am Post subject: |
|
|
Partisan
Joined: 24 Aug 2007
Posts: 365
Location: Columbus, OH
|
| Since you are reading your input messages from a queue how do you know when it is time to close the output files? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Nov 18, 2010 10:32 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| joebuckeye wrote: |
| Since you are reading your input messages from a queue how do you know when it is time to close the output files? |
Either there's an end of sequence indicator or there isn't. If the data is originally coming from a file, it is possible that the end of the file does actually represent an end of sequence - either logically or by assumption. If the file has been turned into a stream of discrete packets and this end of sequence (however it is provided) is then discarded, then it doesn't matter what protocol is used to transfer these discrete packets.
That is to say - the fact that a sequence of records is strung out over MQ does not imply that the sequence does not have a logical termination that is indicated in at least one of the messages. |
|
| Back to top |
|
|
| ata_nitjsr |
| Posted: Fri Nov 19, 2010 1:45 am Post subject: |
|
|
Acolyte
Joined: 08 Apr 2007
Posts: 56
|
| Collection node is not possible i think as it will take from different input & merge it. My input is one queue need to create different file. Can I create temp. Dynamic Q through esql? Then i will create dynamic queue for each type (say queue a, queue b) and put the message there. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Nov 19, 2010 5:06 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| ata_nitjsr wrote: |
| Collection node is not possible i think as it will take from different input & merge it. |
No it won't. Collections are kept separate.
| ata_nitjsr wrote: |
| My input is one queue need to create different file. |
I understand that.
| ata_nitjsr wrote: |
| Can I create temp. Dynamic Q through esql? |
There's no method I'm aware of.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
