| Author |
Message
|
| mqbrk_newbie |
 Posted: Thu May 13, 2010 11:49 pm Post subject: File not processing on FileInput node !! |
 |
|
Acolyte
Joined: 13 Sep 2004
Posts: 70
|
Hello,
I have a large XML file of 30 MB which I am trying to process through FileInput node . I am not able to process such huge file , the file is been sitting on input folder for ever.
When I try processing a 27 MB file it got processed in less than 1 minute.
I have set the java heap size for this execution group as 2 GB.
mqsichangeproperties BrokerName -e EXNAME -o ComIbmJVMManager -n jvmMaxHeapSize -v 2147483648.
I have read some where in this forum that I have to set the environment variables MQSI_FILENODES_MAXIMUM_RECORD_LENGTH , but I assume that is only for the file over 100MB.
Can anyone please advise if I have to change any other configurations to process big files.
we are currently using V6.1
Appreciate your responses in advance. |
|
| Back to top |
|
 |
| fatherjack |
| Posted: Fri May 14, 2010 12:22 am Post subject: |
|
|
Knight
Joined: 14 Apr 2010
Posts: 522
Location: Craggy Island
|
I'd imagine if thebroker was having trouble because of the size of the file you'd see some exception somewhere, rather than the file just sitting there forever.
Other than size does your 27Meg file have exactly the same attributes as the 30meg file.
Specifically, does the filename match the filename / pattern set on the FileInput node? Does its file permissions allow the broker to read/delete/rename/move the file?
_________________
Never let the facts get in the way of a good theory. |
|
| Back to top |
|
|
| mqbrk_newbie |
| Posted: Fri May 14, 2010 7:32 am Post subject: File not processing on FileInput node !! |
|
|
Acolyte
Joined: 13 Sep 2004
Posts: 70
|
Hello fatherjack,
Thanks for your response. Yes Indeed 27Meg File has exactly the same attribute as 30 Meg file . Infact I have renamed both the files with same name. I used chmod 777 to give read , write and execute permission.
I am not sure why it is sitting there for ever.
Is there any other reason that you could think of?
Thanks a lot for your help |
|
| Back to top |
|
|
| fatherjack |
| Posted: Mon May 17, 2010 1:09 am Post subject: |
|
|
Knight
Joined: 14 Apr 2010
Posts: 522
Location: Craggy Island
|
Is the setting of 'Action on successful processing' correct e.g. there isn't a duplicate file in the archive directory preventing the flow from moving the file to the archive directory on completion?
Have you traced the flow to see if any processing is happening?
_________________
Never let the facts get in the way of a good theory. |
|
| Back to top |
|
|
| sushilishere |
| Posted: Sun May 23, 2010 8:04 pm Post subject: |
|
|
Apprentice
Joined: 27 Jul 2008
Posts: 36
|
Just check the eventvwr(If windows) or in messages ( in var/adm -- unix) for any messages. Also check in the reports(if any) whether ur EG has restarted. If it has then u are havin memory issues.
Next resort is to use the trace cmd to get the big trace (mqsichangetrace/mqsireadlog) .
Final resort.. Try performin some action on the input file whem u have placed it in the input folder. If tat action can be performed it means the flow aint even bothered. ( Ideally it shd say the file is in use else ur flow and the file are like Hip hop and Good music .. hahaha ..srry abt tat) |
|
| Back to top |
|
|
|
|
