ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » IBM MQ File Transfer Edition » MFT Locking Files Late

Post new topic  Reply to topic
 MFT Locking Files Late « View previous topic :: View next topic » 
Author Message
PCPXA
PostPosted: Fri Nov 10, 2017 12:32 pm    Post subject: MFT Locking Files Late Reply with quote

Newbie

Joined: 10 Nov 2017
Posts: 5
Does anyone know how to ensure MFT locks a file it's delivering early on in the transfer process? We're having an issue with an MFT delivery over sFTP where the remote end seems to be moving some files off before a lock can be placed on them. We tried sending a batch of 70 files relatively small files (no more than 400KB) over to a client and 5 of them were duplicated as zero bytes files. I say duplicated because the full file does eventually get copied over.

From what we can see in the logs of a trace we ran the zero byte files are not getting to the point where the openAndLockFile parameter is being passed. I've copied a couple of extracts from the log below the first of which shows the transactions of the Zero bytes file and the second cut down one shows the relevant section of a successful delivery:

Zero Byte Transfer:

[09/10/17 18:32:50.606.07] 0033 ProtocolDriver:com.jcraft.jsch.JSch@6a2a1b69
[09/10/17 18:32:50.606.07] 0033 Session:Host:1.1.1.1 UserId:USER1 Server Version:SSH-2.0-Integrated SSH Server (Version 4.0.0.210)
[09/10/17 18:32:50.606.07] 0033 Channel:Status:Connected / Open
[09/10/17 18:32:50.606.07] 0033 HomeDirectory:/
[09/10/17 18:32:50.606.07] 0033 StreamLock:Unlocked
[09/10/17 18:32:50.606.07] 0033 inUse = true]
[09/10/17 18:32:50.606.07] 0033 ]
[09/10/17 18:32:50.606.08] 0033 static c.i.w.bridge.session.BridgeSessionContainerManager ----+--- { releaseSession [FTETransferReference [endPointType=RECEIVER, transferId=414d512055514d424e5930312020202003e3fe5921d40502, wmqConnection=com.ibm.wmqfte.wmqiface.WMQConnectionImpl@558290375 Queue Manager: UQMBNY01, randomDiscEventLogged: false, randomMqDiscForced: false, JmqiEnvironment: com.ibm.mq.jmqi.system.JmqiSystemEnvironment@b6b328df, JmqiMQ: com.ibm.mq.ese.jmqi.ESEJMQI@331b8a89, Hconn: com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D514355514D424E5930312020202003E3FE5921D3ED0B], bridgeFileServerProtocol=null]] <as-before>
[09/10/17 18:32:50.653.00] 0058 @579ff03b c.i.w.b.session.sftp.SFTPSession$SFTPLogger ----+----+----+---- d SFTPLogger data [[FILE]PB-SFTP Connection established]
[09/10/17 18:32:50.747.00] 0058 @579ff03b c.i.w.b.session.sftp.SFTPSession$SFTPLogger ----+----+----+---- d SFTPLogger data [[FILE]PB-SFTP Remote version string: SSH-2.0-Integrated SSH Server (Version 4.0.0.210)]
[09/10/17 18:32:50.747.01] 0058 @579ff03b c.i.w.b.session.sftp.SFTPSession$SFTPLogger ----+----+----+---- d SFTPLogger data [[FILE]PB-SFTP Local version string: SSH-2.0-JSCH-0.1.52]
[09/10/17 18:32:50.747.02] 0058 @579ff03b c.i.w.b.session.sftp.SFTPSession$SFTPLogger ----+----+----+---- d SFTPLogger data [[FILE]PB-SFTP CheckCiphers: aes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc,3des-ctr,arcfour,arcfour128,arcfour256]
[09/10/17 18:32:50.747.03] 0058 @579ff03b c.i.w.b.session.sftp.SFTPSession$SFTPLogger ----+----+----+---- d SFTPLogger data [[FILE]PB-SFTP CheckKexes: diffie-hellman-group14-sha1,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521]
[09/10/17 18:32:50.825.00] 0034 @c4462c49 c.i.w.b.session.sftp.SFTPSession ----+- d getOutputStream data [PB-ADM-WRITESTREAM - successful [remoteFile=/inbound/test/data/FILE.xml]]
[09/10/17 18:32:50.825.01] 0034 @c4462c49 c.i.w.b.session.sftp.SFTPSession ----+ } getOutputStream
[09/10/17 18:32:50.825.02] 0034 @ec1c1e8f c.i.w.io.bridge.BridgeFileChannelImpl ---- } openForWrite
[09/10/17 18:32:50.825.03] 0034 @ec1c1e8f c.i.w.io.bridge.BridgeFileChannelImpl ---- { close [false] [false]
[09/10/17 18:32:50.825.04] 0034 @ec1c1e8f c.i.w.io.bridge.BridgeFileChannelImpl ----+ d close data [Closing output stream CLIENT_SFTP:/inbound/test/data/FILE.xml]
[09/10/17 18:32:51.028.00] 0034 @ec1c1e8f c.i.w.io.bridge.BridgeFileChannelImpl ----+ d close data [Request to close CLIENT_SFTP:/inbound/test/data/FILE.xml]
[09/10/17 18:32:51.028.01] 0034 @c4462c49 c.i.w.b.session.sftp.SFTPSession ----+ { closeFile
[09/10/17 18:32:51.028.02] 0034 @2303dc38 c.i.w.bridge.utils.BridgeStreamLock ----+- { releaseStreamLock
[09/10/17 18:32:51.028.03] 0034 @2303dc38 c.i.w.bridge.utils.BridgeStreamLock ----+-- d releaseStreamLock data [PB-STREAM_REL]
[09/10/17 18:32:51.028.04] 0034 @2303dc38 c.i.w.bridge.utils.BridgeStreamLock ----+- } releaseStreamLock
[09/10/17 18:32:51.028.05] 0034 @c4462c49 c.i.w.b.session.sftp.SFTPSession ----+- d closeFile data [PB-CLOSE - successful]
[09/10/17 18:32:51.028.06] 0034 @c4462c49 c.i.w.b.session.sftp.SFTPSession ----+ } closeFile
[09/10/17 18:32:51.028.07] 0034 @ec1c1e8f c.i.w.io.bridge.BridgeFileChannelImpl ----+ d close data [PB-FTP-CLOSE : [File: CLIENT_SFTP:/inbound/test/data/FILE.xml]]
[09/10/17 18:32:51.028.08] 0034 static c.i.w.bridge.session.BridgeSessionContainerManager ----+ { releaseSession [FTETransferReference [endPointType=RECEIVER, transferId=414d512055514d424e5930312020202003e3fe5921d3f404, wmqConnection=com.ibm.wmqfte.wmqiface.WMQConnectionImpl@-2092937048 Queue Manager: TESTQM , randomDiscEventLogged: false, randomMqDiscForced: false, JmqiEnvironment: com.ibm.mq.jmqi.system.JmqiSystemEnvironment@b6b328df, JmqiMQ: com.ibm.mq.ese.jmqi.ESEJMQI@331b8a89, Hconn: com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D514355514D424E5930312020202003E3FE5921D3FF05], bridgeFileServerProtocol=null]]


Successful Full Transfer:

[09/10/17 18:47:46.283.00] 0032 @543d7676 c.i.w.b.session.sftp.SFTPSession ----+- d getOutputStream data [PB-ADM-WRITESTREAM - successful [remoteFile=//inbound/test/data/FILE.xml]]
[09/10/17 18:47:46.283.01] 0032 @543d7676 c.i.w.b.session.sftp.SFTPSession ----+ } getOutputStream
[09/10/17 18:47:46.283.02] 0032 @72e4e752 c.i.w.io.bridge.BridgeFileChannelImpl ---- } openForWrite
[09/10/17 18:47:46.283.03] 0032 @77c53a87 c.i.w.t.frame.impl.TransferFrameReceiverImpl --- } openAndLockFile returns [c.i.wmqfte.io.bridge.BridgeFileChannelImpl@72e4e752]
[09/10/17 18:47:46.283.04] 0032 @77c53a87 c.i.w.t.frame.impl.TransferFrameReceiverImpl -- } openAndLockFile returns [null] String
[09/10/17 18:47:46.283.05] 0032 @77c53a87 c.i.w.t.frame.impl.TransferFrameReceiverImpl -- { updateChunkStatus [[number=0,lastChunkInTransfer=false,status=RECEIVED,spaceRemaining=0,No Of Slices=1,sliceList=[[fileIndex=0,filePosition=0,buffer=java.nio.HeapByteBuffer[pos=0 lim=100046 cap=262064],lastSliceInFile=false,status=PENDING,missing=false,chunk number=0]]]] [WRITE_PENDING]
[09/10/17 18:47:46.283.06] 0032 @77c53a87 c.i.w.t.frame.impl.TransferFrameReceiverImpl -- } updateChunkStatus
[09/10/17 18:47:46.283.07] 0032 @8b92f5bf c.i.w.t.frame.impl.TransferChunkImpl -- { setStatus [WRITE_PENDING]


Any help will be very much appreciated!
Back to top
View user's profile Send private message
Vitor
PostPosted: Fri Nov 10, 2017 12:37 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
Ah, halting file problem, how I've missed you....
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
toshi78
PostPosted: Fri Dec 15, 2017 1:56 am    Post subject: Reply with quote

Newbie

Joined: 04 May 2012
Posts: 9
Location: Germany
We had a similar problem. The target application was getting the files to fast.

To prevent the receiving application from getting files to early you can put the files into a temporary directory. Then run a post destination command (see Parameters for invoking programs at fteCreateTransfer) to move the file afterwards to the target directory.

Example:
You need to set commandPath=/bin in agent.properties file, restart the agent and execute the fteCreateTransfer
Code:
fteCreateTransfer -sa AGENT1 -da AGENT2 -df '/target/temp/file.txt' -postdst 'executable:/bin/mv(/target/temp/file.txt,/target/file.txt)' /source/file.txt

To be more secure you can do a hardlink of /bin/mv to antoher dedicated directory and use this as commandPath in agent.properties file.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » IBM MQ File Transfer Edition » MFT Locking Files Late
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
  
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.