ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexWebSphere Message Broker SupportIssue with configuring FIle Input node to use SFTP

Post new topicReply to topic
Issue with configuring FIle Input node to use SFTP View previous topic :: View next topic
Author Message
IIBnovice101
PostPosted: Wed Jul 31, 2019 11:22 pm Post subject: Issue with configuring FIle Input node to use SFTP Reply with quote

Newbie

Joined: 31 Jul 2019
Posts: 6

Hi,

I am relatively new to IIB and I am currently having some issues in configuring the File Input Node to use SFTP transfer from a server to a local directory for processing in the message flow..

I have already looked into the previous polls on MQSeries which mention creating a security identity and configurable service. I have done all of this configuration but IIB still fails to pick up the file from the server directory for transferring. I know its not a problem with my permissions or connecting to the server as I can perform transfers via FileZilla with authentication with this server.

The configs I put in the File Input Node are the following :

Remote Transfer : checked
Transfer protocol : SFTP
Server and port : XXX.XXX.XX.XX (have also tried sftp://XXX.XXX.XX.XX, and adding port number :22)
Security identity : myidentity (see below mqsisetdbparms)
Server directory : \\XXX.XXX.XX.XX\General\FilesToTransfer
Transfer mode : binary
Scan delay : 5

In the basic section under directory properties, input directory i put the following :
C:\Users\IIBnovice101\Desktop\testDir\IN and File name or pattern to * (i am trying to get IIB to pick up a .json file in the server to then place it in here)


For the security identity and configurable service I did the following and got successful command completion :

mqsisetdbparms TESTNODE_IIBnovice101 -n sftp::myidentity -u XXXXX -p XXXXX

mqsicreateconfigurableservice TESTNODE_IIBnovice101 -c FtpServer -o TEST1 -n protocol,serverName,scanDelay,remoteDirectory,securityIdentity,cipher,compression,strictHostKeyChecking -v SFTP,XXX.XXX.XX.XX,5,\\XXX.XXX.XX.XX\General\FilesToTransfer,myidentity,blowfish-cbc,9,no

I then performed a restart of the node using mqsistop and mqsistart.


After creating the configurable service I also tried to fill out the FTP section as follows :

Remote Transfer : checked
Transfer protocol : SFTP
Server and port : TEST1 (from my understanding you only need to specify this as it overrides all the hardcoded config in this section)

But still no success in getting IIB to pick up the file.....

Any ideas or input as to why?

Thank you in advance for any help!!

Best regards,

Philippe
Back to top
View user's profile Send private message
abhi_thri
PostPosted: Thu Aug 01, 2019 12:30 am Post subject: Reply with quote

Partisan

Joined: 17 Jul 2017
Posts: 312
Location: UK

hi...what happens when you starts the flow, is it able to connect to the remote server and poll for the file? you can check this by looking for any errors in the syslog when the flow got started (also can use netstat to see whether a connection is established)
Back to top
View user's profile Send private message
IIBnovice101
PostPosted: Thu Aug 01, 2019 12:57 am Post subject: Reply with quote

Newbie

Joined: 31 Jul 2019
Posts: 6

Hi! Thank you for your reply. Nothing seems to happen when i start the flow...i have added a Trace node to my File Input node with the following config :

C:\Users\IIBnovice101\Desktop\log\Error.log

Time: ${CURRENT_TIMESTAMP}
Broker: ${BrokerName} Version: ${BrokerVersion} Platform: ${Family}
ProcessID: ${ProcessId} BrokerUserId: ${BrokerUserId}
ExecutionGroupLabel: ${ExecutionGroupLabel}
Root:
${Root}
LocalEnvironment:
${LocalEnvironment}
ExceptionList:
${ExceptionList}

But I do not see any change in it so it seems the error is occurring before the flow. I will look into netstat for the connection, but as i previously mentioned the connection seems to work as i can connect to the server via FileZilla, and also when i specified the server directory as input directory (just to test) in the "Basic" section of the file input node it goes to fetch the file without any problem, but this is not via SFTP since it fetches it regardless of whether the FTP box is checked or not..
Back to top
View user's profile Send private message
abhi_thri
PostPosted: Thu Aug 01, 2019 1:26 am Post subject: Reply with quote

Partisan

Joined: 17 Jul 2017
Posts: 312
Location: UK

IIBnovice101 wrote:
Nothing seems to happen when i start the flow..


hi...well, something is happening when you start the flow...it either manages to connect to the remote server or it doesn't. If your broker is running on Windows the errors should get logged at Windows Event Viewer->Windows Logs-> Application. Alternatively you could enable user trace for the flow and restart it and see what the trace says.
Back to top
View user's profile Send private message
IIBnovice101
PostPosted: Thu Aug 01, 2019 1:48 am Post subject: Reply with quote

Newbie

Joined: 31 Jul 2019
Posts: 6

Ah yes I see it now, i just went into event logs and found that there is indeed the following error :

( TESTNODE_IIBnovice101.default ) File node ''File Input'' in message flow ''XEflow'' could not connect to remote ''SFTP'' server ''XXX.XXX.XX.XX''. Reason: ''Error while moving to directory: \\XXX.XXX.XX.XX\General\FilesToTransfer. Reason: No such file'.'

The FileInput or FileOutput node failed to open a connection to a remote server. Possible reasons for this failure are that the server could not be contacted, the initial FTP protocol was not observed, or an error situation exists at the server. The reason text is from the detected connection failure.

Ensure that:
- The server address is correct and is locally resolved to a valid IP address.
- The server port is correctly specified (the default port for FTP is 21, and for SFTP it is port 22).
- A firewall does not block access to the remote server.

Any ideas how i could solve/further investigate this?

I have made sure that the server address and port is indeed correct, the directory folders are existing and as far as i know there is no firewall blocking access..
Back to top
View user's profile Send private message
abhi_thri
PostPosted: Thu Aug 01, 2019 2:38 am Post subject: Reply with quote

Partisan

Joined: 17 Jul 2017
Posts: 312
Location: UK

IIBnovice101 wrote:
Error while moving to directory: \\XXX.XXX.XX.XX\General\FilesToTransfer. Reason: No such file'.'


this is the error, that the Broker failed to move to the specified directory...the directory path '\\XXX.XXX.XX.XX\General\FilesToTransfer' looks wrong to me as it shouldn't contain the hostname
Back to top
View user's profile Send private message
IIBnovice101
PostPosted: Thu Aug 01, 2019 3:26 am Post subject: Reply with quote

Newbie

Joined: 31 Jul 2019
Posts: 6

Thanks for pointing that out. I indeed switched the directory format in the server directory field to the following /D:/General/FilesToTransfer, and i no longer get the error. For testing I specified a folder that didnt exist yet on the server in the server directory field and it created it without a problem. However, when i place my .json file in the folder directory specified below IIB still doesn't seem to pick it up despite me specifying a security identity profile and restarting the node.

Any idea how I could further investigate this?

My setup is now the following :

Remote Transfer : Checked
Transfer protocol : SFTP
Sever and port : XXX.XXX.XX.XX
Security identity : myidentity
Server directory : /D:/General/FilesToTransfer.

Thanks a lot for your help until now btw, i really appreciate it!
Back to top
View user's profile Send private message
abhi_thri
PostPosted: Thu Aug 01, 2019 3:56 am Post subject: Reply with quote

Partisan

Joined: 17 Jul 2017
Posts: 312
Location: UK

IIBnovice101 wrote:
I specified a folder that didnt exist yet on the server in the server directory field and it created it without a problem.


This shows that Broker is now fully connected to the remote server and is polling for the file pattern you specified. As you've specified * ideally the file should get picked up (do check the Event viewer for any errors), also verify whether the file owner/permissions. The broker user id need full permissions on the file as it need to delete the file after reading.
Back to top
View user's profile Send private message
IIBnovice101
PostPosted: Thu Aug 01, 2019 4:07 am Post subject: Reply with quote

Newbie

Joined: 31 Jul 2019
Posts: 6

Indeed I was expecting it to get picked up but for some reason it is still not doing so. The file i am currently trying to process has no permissions attached to it and i have full access rights on the server. The event viewer doesnt show any errors either when i deploy...

In any case I will just trial and error and hopefully it will eventually work, at least i now know that IIB is indeed fully connected to the server. Thanks a lot for your quick responses and very useful tips!!
Back to top
View user's profile Send private message
abhi_thri
PostPosted: Thu Aug 01, 2019 4:26 am Post subject: Reply with quote

Partisan

Joined: 17 Jul 2017
Posts: 312
Location: UK

IIBnovice101 wrote:
The file i am currently trying to process has no permissions attached to it and i have full access rights on the server.


hi...you might've full access to the file but what about the broker user id, are you running broker under your user id or some generic id (you can check this in Windows at the user under which the broker service is running at services.msc)
Back to top
View user's profile Send private message
IIBnovice101
PostPosted: Mon Aug 05, 2019 1:23 am Post subject: Reply with quote

Newbie

Joined: 31 Jul 2019
Posts: 6

In the end it was indeed just a permissions problem, thank you so much for your help!
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Page 1 of 1

MQSeries.net Forum IndexWebSphere Message Broker SupportIssue with configuring FIle Input node to use SFTP
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.