| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| File nodes in MB v6.1 and Large files(nearly 20 MB) |
« View previous topic :: View next topic » |
| Author |
Message
|
| ammu |
 Posted: Thu Oct 09, 2008 9:11 pm Post subject: File nodes in MB v6.1 and Large files(nearly 20 MB) |
 |
|
Apprentice
Joined: 26 Apr 2007
Posts: 25
|
Hi,
Can someone please clarify a doubt. I thought of using File nodes provided by MB v6.1 for transferring some files. Later I came to know that these files in actaul environment can vary upto 20 MB.
Is it like the Filenodes load the entire file into the MB memory, or will it just initiates the FTP process, so that memory is not at all consumed?
1)If the Memory is consumed, definetely it will cause issues for large files. One option whihc I coukld find in the IBM documentaion is that to split the file into chunks and then transmit.
2)Is there a any other option that could help me in this regard.
Someone please clarify.
Thanks,
Ammu. |
|
| Back to top |
|
 |
| Gaya3 |
| Posted: Thu Oct 09, 2008 9:31 pm Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
Read this
http://publib.boulder.ibm.com/infocenter/wmbhelp/v6r1m0/index.jsp?topic=/com.ibm.etools.mft.doc/ac55150_.htm
---
If you specified Fixed Length in Record detection, use Length to specify the required length of the output record. This value must be between 1 byte and 100 MB. The default is 80 bytes.
If you specify Whole File, Fixed Length, or Delimited in Record detection, a limit of 100 MB applies to the length of the records.
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| ammu |
| Posted: Thu Oct 09, 2008 10:18 pm Post subject: |
|
|
Apprentice
Joined: 26 Apr 2007
Posts: 25
|
Thanks Gayathri, for the quick reply.
So anyways Broker [b]has to read the file(whether it is in small chunks or as whole file) if I'm using FileInput node. So it has to consume the memory.Am I right?[/b]
In my scenario, it is not mandatory that MB has to do this huge File transfer. It can be done through a shell scripts/some custom applciations itself. So only if the Broker machine memory is not consumed, We can avoid this customapp overhead and use the existing middleware itself. I'm checking on that portion mainly.
Even if, I'm going splitting the file into chunks and transfer, it can cause issues apparently, rite? Something like, out of 'n' chunks, one chunk faced issues in file transfer, rite?
regards,
Ammu. |
|
| Back to top |
|
|
| Gaya3 |
| Posted: Thu Oct 09, 2008 11:24 pm Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
1. limit the input file size to less than 100 MB (here its 20 MB)
2. whole file and and transfering file in small chunks indeed there will be a difference in memory management.
3.It uses the RAM memory, for storing the input and output files (Whole file category),
4. incase if you want multiple output files at the destination side, then it will eat the memory again
5. Suggest you to make it as Small chunks - RAM memory usage will be minimal
6. You can even think of MQ FTP features to transfer the file, if it doesnt have any transformation.
7. dont transfer this type of files, in the peak interval of MB transformation
8. I dont think so there will be any discrepany while using small chunks (have to be tested, or you can wait for experts answer on this)
There are lot of features if you are using MB.
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
