| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Parse large TDS files to XML |
« View previous topic :: View next topic » |
| Author |
Message
|
| fszostak |
 Posted: Mon Feb 14, 2011 3:55 am Post subject: Parse large TDS files to XML |
 |
|
Acolyte
Joined: 09 Feb 2011
Posts: 64
Location: Curitiba, Brazil
|
Hi all!
I have a doubt... have a large cobol data file (around 7Mb).
File Content sample:
| Code: |
00AAAAAAAAAAAAAAAAAAAAAA111111111111
10BBBBBBBBBBBBBBBBBBBBBCCCCCCCCCCCCCBBBBBBBB
10BBBBBBBBBBBBBBBBBBBBBCCCCCCCCCCCCCBBBBBBBB
20DDDDEEEE00000000000
20DDDDEEEE00000000000
20DDDDEEEE00000000000
10BBBBBBBBBBBBBBBBBBBBBCCCCCCCCCCCCCBBBBBBBB
10BBBBBBBBBBBBBBBBBBBBBCCCCCCCCCCCCCBBBBBBBB
10BBBBBBBBBBBBBBBBBBBBBCCCCCCCCCCCCCBBBBBBBB
.
.
|
i solve the parse this file with a TDS structure, with Mapping node i solve the XML transformation, very simple flow, great solution.
FileInput -> Mapping -> FileOutput
| Code: |
<root>
<header>
<f1>AAAAAAAAAAAAAAAAAAAAAA</f1>
<f2>111111111111</f2>
</header>
<body>
<register>
<f1>BBBBBBBBBBBBBBBBBBBBB</f1>
<f2>CCCCCCCCCCCCC</f2>
<f3>BBBBBBBB</f3>
<register>
<register>
<f1>BBBBBBBBBBBBBBBBBBBBB</f1>
<f2>CCCCCCCCCCCCC</f2>
<f3>BBBBBBBB</f3>
<f4_regs>
<f4>
<f4_1>DDDD</f4_1>
<f4_2>EEEE</f4_2>
<f4_3<00000000000</f4_3>
</f4>
<f4>
<f4_1>DDDD</f4_1>
<f4_2>EEEE</f4_2>
<f4_3<00000000000</f4_3>
</f4>
<f4>
<f4_1>DDDD</f4_1>
<f4_2>EEEE</f4_2>
<f4_3<00000000000</f4_3>
</f4>
<f4_regs>
</register>
.
.
.
</body>
</root>
|
But i think if parse all file content in InputFile node, this can generate a high consumption of memory or not?
I can to parse "register by register" but as I do to take root/header/body tags control?
Thank yiu for your answers
Szostak |
|
| Back to top |
|
 |
| kimbert |
| Posted: Mon Feb 14, 2011 5:12 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Feb 14, 2011 5:14 am Post subject: Re: Parse large TDS files to XML |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| fszostak wrote: |
| I have a doubt... have a large cobol data file (around 7Mb). |
You and I have different definitions of "large"! 
| fszostak wrote: |
| But i think if parse all file content in InputFile node, this can generate a high consumption of memory or not? |
There's a number of discussions in this forum (and some IBM whitepapers) on handling large data structures, all of which it's too early in the morning for me to be looking for and all of which deal with rather larger structures than 7Mb!
The search facility is your friend here.
Out of interest, what OS, what version of WMB and how much memory on the box?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fszostak |
| Posted: Mon Feb 14, 2011 2:56 pm Post subject: Re: Parse large TDS files to XML |
|
|
Acolyte
Joined: 09 Feb 2011
Posts: 64
Location: Curitiba, Brazil
|
| Vitor wrote: |
| fszostak wrote: |
| I have a doubt... have a large cobol data file (around 7Mb). |
You and I have different definitions of "large"!
| fszostak wrote: |
| But i think if parse all file content in InputFile node, this can generate a high consumption of memory or not? |
There's a number of discussions in this forum (and some IBM whitepapers) on handling large data structures, all of which it's too early in the morning for me to be looking for and all of which deal with rather larger structures than 7Mb!
The search facility is your friend here.
Out of interest, what OS, what version of WMB and how much memory on the box? |
I understand. I also think that the file is not large.
I have set up the TDS parser to have the ability to read all records at once, then I think that this process may require more resources, so a "message" would contain 7Mb. Sorry, maybe my explanation was not clear .
I am tending to read the file line by line, but will I need to write the start and end tags of XML in a compute node. It's no good!
I work with WMB6.1.005 and have 16Gb of RAM, but its sharing with anothers "execution groups".
Thanks!
Szostak |
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Feb 14, 2011 3:35 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| I am tending to read the file line by line, but will I need to write the start and end tags of XML in a compute node. It's no good! |
Not true. Please read my post earlier in this thread. |
|
| Back to top |
|
|
| fszostak |
| Posted: Mon Feb 14, 2011 4:49 pm Post subject: |
|
|
Acolyte
Joined: 09 Feb 2011
Posts: 64
Location: Curitiba, Brazil
|
| kimbert wrote: |
| Quote: |
| I am tending to read the file line by line, but will I need to write the start and end tags of XML in a compute node. It's no good! |
Not true. Please read my post earlier in this thread. |
Hi Kimbert,
Sorry! I read your post now.
Szostak
_________________
WMB 6.1.005 |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
