| Author |
Message
|
| jrsdm |
 Posted: Wed Oct 24, 2012 5:13 pm Post subject: Message broker file input node process Large XML file |
 |
|
Apprentice
Joined: 24 Oct 2012
Posts: 27
|
Hi ,
I am doing some POC. The requirement is to process the large xml file and do the transformation and create different files based on some condition send it to the different remote directory.
I am using FIN->Compute->FOT node but not able to succeed.
Could anyone advice me is the this possible to pull the 50 GB of XML file using fileinput node ? |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Wed Oct 24, 2012 5:38 pm Post subject: Re: Message broker file input node process Large XML file |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| jrsdm wrote: |
Hi ,
I am doing some POC. The requirement is to process the large xml file and do the transformation and create different files based on some condition send it to the different remote directory.
I am using FIN->Compute->FOT node but not able to succeed.
Could anyone advice me is the this possible to pull the 50 GB of XML file using fileinput node ? |
It is but you have to split the file by its atomic transactions...
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| jrsdm |
| Posted: Wed Oct 24, 2012 5:47 pm Post subject: |
|
|
Apprentice
Joined: 24 Oct 2012
Posts: 27
|
Thanks Saper.
Could you give me some small hint on the same , so that I can do the research in right direction. |
|
| Back to top |
|
|
| marko.pitkanen |
| Posted: Wed Oct 24, 2012 10:31 pm Post subject: |
|
|
Chevalier
Joined: 23 Jul 2008
Posts: 440
Location: Jamsa, Finland
|
Hi,
You should perhaps research what you can do with FIN's "Record detection" property. But you should perhaps consider carefully should this this requirement be solved (only) with broker.
--
marko |
|
| Back to top |
|
|
| Esa |
| Posted: Thu Oct 25, 2012 12:09 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| Check the Large Message Prosessing sample in the Toolkit, too. In case your splitting rule is complex. |
|
| Back to top |
|
|
|
|
