ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexWebSphere Message Broker Support[Closed] ACE11 File Input node not processing file from NFS

Post new topicReply to topic
[Closed] ACE11 File Input node not processing file from NFS View previous topic :: View next topic
Author Message
mqsi_man
PostPosted: Thu Feb 21, 2019 9:33 pm Post subject: [Closed] ACE11 File Input node not processing file from NFS Reply with quote

Novice

Joined: 28 Aug 2013
Posts: 19

Hello,

My application contains a message flow that reads a File, record by record, using File Input node and writes transformed records to output file using File Output node. Functionality works fine on windows local file system but now, I deployed the application to process file from shared location location on network and strangely, application is able to write output file to shared location but not able to read input file from it.

I have verified that Input / Output directories 777 permissions.
verified that filename pattern and input file name are correct.

For File Input node, mqsibackout and mqsitransitin directories get created on NFS location and when I place the file, I see 0 KB file under mqsitransitin/locks which tells me that ACE detected the presence of the file.

User trace shows following,

2019-02-21 11:58:51.771884 984 UserTrace BIP11504I: Waiting for data from input node 'File Input'.
A thread is waiting for data from input node 'File Input' in flow 'A.B.C'.


ace version

Version: '11.0.0.3'
Product: 'IBM App Connect Enterprise'
Build Number: '16552'

NFS shared mount has following settings,

/nfs type xfs (rw,relatime,seclabel,attr2,inode64,noquota)

Appreciate any help.

Thanks.

I had found a similar post on MQSeries, http://mqseries.net/phpBB/viewtopic.php?t=69831&sid=84d962db64978db430e75ffdea38eebf and I am running ACE under my LAN id which is what I am using to access NFS.


Last edited by mqsi_man on Wed Feb 27, 2019 3:52 am; edited 1 time in total
Back to top
View user's profile Send private message
mqsi_man
PostPosted: Fri Feb 22, 2019 6:43 am Post subject: Reply with quote

Novice

Joined: 28 Aug 2013
Posts: 19

Update : Service trace shows following,
Code:

2019-02-22 14:01:09.966810     8144             com.ibm.broker.nodes.filenodes.InputFileQueue@78582ae9.getNext, 'Polling file InputFile[\\<host name>\<path>\<file name>]'
2019-02-22 14:01:09.967062     8144             ComIbmFileInputNode.getNextLocalFile, 'Trying to create a lockfile for file: ', '\\<host name>\<path>\<file name>'
2019-02-22 14:01:09.980007     8144             ComIbmFileInputNode.getNextLocalFile, 'Attempt to lock lockfile: ', '\\<host name>\<path>\<file name>'
2019-02-22 14:01:09.981101     8144             ComIbmFileInputNode.getNextLocalFile, 'This process doesn't own the lock file - refusing to delete it'
2019-02-22 14:01:09.987349     8144             com.ibm.broker.nodes.filenodes.ComIbmFileInputNode@-2c8fb14a.readData, 'java.io.IOException: The request is not supported. at sun.nio.ch.FileDispatcherImpl.lock0(Native Method) at sun.nio.ch.FileDispatcherImpl.lock(FileDispatcherImpl.java:115) at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1137) at com.ibm.broker.nodes.filenodes.AbstractFileInputNode.getNextLocalFile(AbstractFileInputNode.java:715) at com.ibm.broker.nodes.filenodes.AbstractFileInputNode.getNextFile(AbstractFileInputNode.java:826) at com.ibm.broker.nodes.filenodes.AbstractFileInputNode.readData(AbstractFileInputNode.java:1017) '
2019-02-22 14:01:09.987524     8144         } com.ibm.broker.nodes.filenodes.ComIbmFileInputNode@-2c8fb14a.readData, 'READDATA_TIMED_OUT_WAIT_REQUIRED'
2
[/code]
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Fri Feb 22, 2019 6:54 am Post subject: Reply with quote

Grand Poobah

Joined: 18 Nov 2003
Posts: 19945
Location: LI,NY

Are you sure the userid running the integration service has read rights to the file in the directory?? Looks to me the file input node can see the file name but not the file content... Ideally the ACE service id should have full rights to the file...
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
juandelcid
PostPosted: Sat Feb 23, 2019 12:05 pm Post subject: Reply with quote

Newbie

Joined: 22 Feb 2019
Posts: 6

mqsi_man wrote:

Code:

ComIbmFileInputNode.getNextLocalFile, 'Trying to create a lockfile for file: ', '\\<host name>\<path>\<file name>'
2019-02-22 14:01:09.980007     8144             ComIbmFileInputNode.getNextLocalFile, 'Attempt to lock lockfile: ', '\\<host name>\<path>\<file name>'
2019-02-22 14:01:09.981101     8144             ComIbmFileInputNode.getNextLocalFile, 'This process doesn't own the lock file - refusing to delete it'
2019-02-22 14:01:09.987349     8144             com.ibm.broker.nodes.filenodes.ComIbmFileInputNode@-2c8fb14a.readData, 'java.io.IOException: The request is not supported. at sun.nio.ch.FileDispatcherImpl.lock0(Native Method) at sun.nio.ch.FileDispatcherImpl.lock(FileDispatcherImpl.java:115) at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1137) at com.ibm.broker.nodes.filenodes.AbstractFileInputNode.getNextLocalFile(AbstractFileInputNode.java:715) at com.ibm.broker.nodes.filenodes.AbstractFileInputNode.getNextFile(AbstractFileInputNode.java:826) at com.ibm.broker.nodes.filenodes.AbstractFileInputNode.readData(AbstractFileInputNode.java:1017) '
2019-02-22 14:01:09.987524     8144         } com.ibm.broker.nodes.filenodes.ComIbmFileInputNode@-2c8fb14a.readData, 'READDATA_TIMED_OUT_WAIT_REQUIRED'



I see READDATA_TIMED_OUT_WAIT_REQUIRED.

Have a look at this:

https://serverfault.com/questions/61594/what-does-no-locks-available-mean

It says that the lockd might need to be started/restarted.

-JG
Back to top
View user's profile Send private message
mqsi_man
PostPosted: Sun Feb 24, 2019 11:09 pm Post subject: Reply with quote

Novice

Joined: 28 Aug 2013
Posts: 19

fjb_saper wrote:
Are you sure the userid running the integration service has read rights to the file in the directory?? Looks to me the file input node can see the file name but not the file content... Ideally the ACE service id should have full rights to the file...


It is my best guess as file permissions are 777 and it is owned by "nfsnobody" and this is what nfs export looks like,

[root@host In]# more /etc/exports
/nfs *(rw,all_squash,anonuid=65534,anongid=65534)

Basically, I am trying to give all users all access (read/write/delete) and clearly, I am missing something.
Back to top
View user's profile Send private message
mqsi_man
PostPosted: Sun Feb 24, 2019 11:14 pm Post subject: Reply with quote

Novice

Joined: 28 Aug 2013
Posts: 19

juandelcid wrote:
mqsi_man wrote:

Code:

ComIbmFileInputNode.getNextLocalFile, 'Trying to create a lockfile for file: ', '\\<host name>\<path>\<file name>'
2019-02-22 14:01:09.980007     8144             ComIbmFileInputNode.getNextLocalFile, 'Attempt to lock lockfile: ', '\\<host name>\<path>\<file name>'
2019-02-22 14:01:09.981101     8144             ComIbmFileInputNode.getNextLocalFile, 'This process doesn't own the lock file - refusing to delete it'
2019-02-22 14:01:09.987349     8144             com.ibm.broker.nodes.filenodes.ComIbmFileInputNode@-2c8fb14a.readData, 'java.io.IOException: The request is not supported. at sun.nio.ch.FileDispatcherImpl.lock0(Native Method) at sun.nio.ch.FileDispatcherImpl.lock(FileDispatcherImpl.java:115) at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1137) at com.ibm.broker.nodes.filenodes.AbstractFileInputNode.getNextLocalFile(AbstractFileInputNode.java:715) at com.ibm.broker.nodes.filenodes.AbstractFileInputNode.getNextFile(AbstractFileInputNode.java:826) at com.ibm.broker.nodes.filenodes.AbstractFileInputNode.readData(AbstractFileInputNode.java:1017) '
2019-02-22 14:01:09.987524     8144         } com.ibm.broker.nodes.filenodes.ComIbmFileInputNode@-2c8fb14a.readData, 'READDATA_TIMED_OUT_WAIT_REQUIRED'



I see READDATA_TIMED_OUT_WAIT_REQUIRED.

Have a look at this:

https://serverfault.com/questions/61594/what-does-no-locks-available-mean

It says that the lockd might need to be started/restarted.

-JG


ps -ef | grep -i lockd
root 28 2 0 Feb22 ? 00:00:00 [kblockd/0]
root 29 2 0 Feb22 ? 00:00:00 [kblockd/1]
root 34484 2 0 08:30 ? 00:00:00 [lockd]
root 34642 33569 0 09:10 pts/0 00:00:00 grep -i lockd

Restarted but no luck.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Mon Feb 25, 2019 5:52 am Post subject: Reply with quote

Grand Poobah

Joined: 18 Nov 2003
Posts: 19945
Location: LI,NY

Remind us, which version of NFS are we talking about here?
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
mqsi_man
PostPosted: Mon Feb 25, 2019 8:59 pm Post subject: Reply with quote

Novice

Joined: 28 Aug 2013
Posts: 19

fjb_saper wrote:
Remind us, which version of NFS are we talking about here?


[root@host/]# nfsstat
Server rpc stats:
calls badcalls badclnt badauth xdrcall
381825 102735 0 102735 0

Server nfs v2:
null getattr setattr root lookup readlink
12 100% 0 0% 0 0% 0 0% 0 0% 0 0%
read wrcache write create remove rename
0 0% 0 0% 0 0% 0 0% 0 0% 0 0%
link symlink mkdir rmdir readdir fsstat
0 0% 0 0% 0 0% 0 0% 0 0% 0 0%

Server nfs v3:
null getattr setattr lookup access readlink
21 0% 381122 99% 38 0% 226 0% 45 0% 0 0%
read write create mkdir symlink mknod
12 0% 14 0% 21 0% 14 0% 0 0% 0 0%
remove rmdir rename link readdir readdirplus
17 0% 9 0% 9 0% 0 0% 65 0% 102 0%
fsstat fsinfo pathconf commit
66 0% 7 0% 7 0% 18 0%



Back to top
View user's profile Send private message
fjb_saper
PostPosted: Tue Feb 26, 2019 5:27 am Post subject: Reply with quote

Grand Poobah

Joined: 18 Nov 2003
Posts: 19945
Location: LI,NY

What is the outcome if you were using NFS4 ? 2 and 3 have some limitations where file locking is concerned...
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
mqsi_man
PostPosted: Tue Feb 26, 2019 6:17 am Post subject: Reply with quote

Novice

Joined: 28 Aug 2013
Posts: 19

fjb_saper wrote:
What is the outcome if you were using NFS4 ? 2 and 3 have some limitations where file locking is concerned...


You beat me to it. Per IBM's documentation, File nodes must use NFS v4 as NFS v3 doesn't support/has issues locking of file over network.

So I configured NFS mount with NFSv4 server and turns out, Windows doesn't support NFSv4

At this time, I am working to mount the NFSv4 share inside a docker container in Linux VM and I am hopeful that it will work.

More to come.
Back to top
View user's profile Send private message
mqsi_man
PostPosted: Wed Feb 27, 2019 3:50 am Post subject: Reply with quote

Novice

Joined: 28 Aug 2013
Posts: 19

Issue was due to NFS v3. We switched to NFS v4 server and are now able to successfully process files using File Input node.

Thanks fjb_saper and juan!
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Wed Feb 27, 2019 5:59 am Post subject: Reply with quote

Grand Poobah

Joined: 18 Nov 2003
Posts: 19945
Location: LI,NY

mqsi_man wrote:
fjb_saper wrote:
What is the outcome if you were using NFS4 ? 2 and 3 have some limitations where file locking is concerned...


You beat me to it. Per IBM's documentation, File nodes must use NFS v4 as NFS v3 doesn't support/has issues locking of file over network.

So I configured NFS mount with NFSv4 server and turns out, Windows doesn't support NFSv4

At this time, I am working to mount the NFSv4 share inside a docker container in Linux VM and I am hopeful that it will work.

More to come.


For windows, have you tried CIFS ?
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:
Post new topicReply to topic Page 1 of 1

MQSeries.net Forum IndexWebSphere Message Broker Support[Closed] ACE11 File Input node not processing file from NFS
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.