| Author |
Message
|
| Laurens |
 Posted: Thu Oct 04, 2012 1:44 am Post subject: File Input using FTP : need to fetch specific files |
 |
|
Apprentice
Joined: 01 Oct 2009
Posts: 35
|
Hi ,
I try to implement the following requirement :
Broker receives a request message with in it the name of a file to be picked up from an FTP site. The name changes with each request message.
My problem is that the FileInput node can pickup via ftp but cannot dynamically specify the file to be picked up. (only patterns of fixed name)
The FileRead node can pick up a specific file but cannot do FTP (unless I'm mistaken)
Pumping all data from the ftp site to a local dir and then pickup through fileread node is not an option
Any ideas ?
Kindest regards
Laurens |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Thu Oct 04, 2012 2:03 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Yes, your requirement as stated can't be used with Broker.
FTE can do it, I believe, although you'll have to do a bit of complicated work to create an FTE request sent via the FTE nodes.
Otherwise you get to write .NETCompute or JavaCompute code to execute the FTP yourself. |
|
| Back to top |
|
|
| McueMart |
| Posted: Thu Oct 04, 2012 2:04 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
| One option is to write custom code in a JavaComputeNode to pull the file over FTP. Obviously not the ideal solution so im sure some others will be along with other ideas. |
|
| Back to top |
|
|
| McueMart |
| Posted: Thu Oct 04, 2012 2:08 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
| If you *had* to use WMB to do this using native nodes, I suppose you could use the CMP API to dynamically configure a configurable service, and then invoke a restart on an EG (to cause the new configurable service properties to be picked up). Restarting an EG every time you need to pick up a new file doesnt seem like an awesome design though. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Oct 04, 2012 2:12 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| McueMart wrote: |
| If you *had* to use WMB to do this using native nodes, I suppose you could use the CMP API to dynamically configure a configurable service, and then invoke a restart on an EG (to cause the new configurable service properties to be picked up). Restarting an EG every time you need to pick up a new file doesnt seem like an awesome design though. |
This is an excellently bad suggestion, very well done!
The v8 Broker API starts to let you actually create new flows, so one could potentially even just write and deploy a new flow that had a FileInput with the requested location... But again, it's a really terrible idea and nobody should do it, it's only slightly better than changing a configurable service because it doesn't require you restart the EG. |
|
| Back to top |
|
|
| McueMart |
| Posted: Thu Oct 04, 2012 2:21 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
| Quote: |
| This is an excellently bad suggestion, very well done! |
I do try my best! |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Oct 04, 2012 2:35 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| McueMart wrote: |
| Quote: |
| This is an excellently bad suggestion, very well done! |
I do try my best! |
 |
|
| Back to top |
|
|
| Esa |
| Posted: Thu Oct 04, 2012 5:23 am Post subject: Re: File Input using FTP : need to fetch specific files |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| Laurens wrote: |
Hi ,
I try to implement the following requirement :
Broker receives a request message with in it the name of a file to be picked up from an FTP site. The name changes with each request message.
My problem is that the FileInput node can pickup via ftp but cannot dynamically specify the file to be picked up. (only patterns of fixed name)
The FileRead node can pick up a specific file but cannot do FTP (unless I'm mistaken)
Pumping all data from the ftp site to a local dir and then pickup through fileread node is not an option
Any ideas ?
Kindest regards
Laurens |
Laurens, you need to change your design, turn it upside down. Instead of just sending a message with a file name your sender system also needs to move or copy the file into the directory your FileInput node is listening to.
Obviously the trigger message contains more data than just the file name. If that is the case, you need somehow to correlate the message with the correct message so that you can use MQGet to pick up the correct message and process the file and the message together. The best solution would be to generate filenames that can be used as MQ correlation identifiers. Or to be put in MQMD.ApplIdentityDatam in which case you will need to browse and select the correct message in your flow. In the worst case you need to browse the messages, parse them to be able to access the file name field and then use MQGet to remove the selected message from the queue.
How does this sound? |
|
| Back to top |
|
|
| Laurens |
| Posted: Mon Oct 08, 2012 12:14 am Post subject: |
|
|
Apprentice
Joined: 01 Oct 2009
Posts: 35
|
Hey Esa , sounds very ingenious but unfortunately not possible in my context.
The way of working is dictated by the electricity market in the BENELUX
There is one main organisation that distributes file data between the partners.
Each partner need to calls a webservice that will reply if and which file is available. If so it has to pick the file up from a remote location through sftp.
I think I will simply rename the file on the remote location with a particular prefix. The fileinput node will then pick it up since it will match the pattern configured on the node.
Hope I didn't miss somthing with this idea.
Kindest regards
Laurens Raven |
|
| Back to top |
|
|
|
|
