Sporadically corrupt files after submitting with sFTP component

0
Hi For a project, I have implemented the sFTP component in a sFTP client that transfer files from and to a sFTP server. Unfortunately, I don't have any meaningful support from the sFTP server side (no competent contact). The transferred files are .xml files. Principally the transfer work stable: - more than 15'000 files were transferred **from** the sFTP server yet **-> no corrupt files** - more than 26'000 files were transferred **to** the sFTP server yet **-> three corrupt files** So from a total of more than 41'000 files, three files were stored corrupt on the sFTP server after transfer. (that shows, that it is not a principal problem) Process (send files): The .xml files are exported by another application (over a timer). The sFTP client (with the sFTP component): - query a directory for new files over a timer - reads new file names in an array (in memory) - copy all new exported files to the sFTP server - checks, if the files are transferred and - if yes - moves the files to an \archive\ directory Details to the error: The corrupt stored files are exported correct (the complete file content is stored in the file) The files are transferred normally (without sFTP error) and the moved to \archive\ as they were transferred. On the sFTP server the files are stored **corrupt (the end of the .xml file is missing)** **This then causes a stop of the whole import queue on the sFTP server (what means, the whole interface is dead)** Note: For me a clear design error in the import interface, but - as I wrote - I have no support on the sFTP server, so... no chance, that this will be changed... Nevertheless, I (unfortunately) now have to try, to solve the problem... Questions: **- is this a known problem (sporadically corrupt files)?** **- can it be, that the sFTP library start to transfer a file, although the file is not written completely by the export application?** **- can it be, that the sFTP library cannot write the file completely on the sFTP server, as the file is opened on the sFTP Server, before it is written completely (whereby, I think, this should cause a crash of the transfer...?)** **- do you have some other ideas, what can cause the problems?** Note: As I don't have any support from the sFTP server side, it's no solution to check the file size / checksum after submission and simply send the file once again, if not equal (as the file is read as soon it is stored on the sFTP server). **Thanks for meaningful answers to my questions above...** Fredy
edited 5/18/2018 10:01:38 AM
asked 5/18/2018 10:01:13 AM
add a comment

2 Answers

0
First, I am disappointed with the support, as I did not receive any substantiated answers here in the forum... I have investigated the problem in deep in the meantime myself and post the results here **for other users**... For calming to all... it seems that it wasn’t a problem with the component pro library... It seems as the sftp client has took some files (very randomly) for transfer, **before they were written completely** (what should not be possible technical in my understanding, but seems to be the case). I now have implemented special functions to secure, that this can’t happen in future: - I now try to open the ”to transfer” file exclusive in the sFTP client, before it is transferred to the sFTP server (in a try-catch block) - If this is not possible (= the catch block is triggered), the file is not transferred (the file then is transferred with the next “timer tick” - in my case after 15 seconds) I was able to comprehend, **that** the special functions have been triggered (and have worked -> no corrupt files) and hope, that the problem is solved now. So if you have a similar scenario (export files with program A and transfer it with program B), it may be a good idea, to implement similar functions (as the files can be transferred before they are written completely). Hope this helps someone...
 
answered 6/22/2018 2:15:04 PM
add a comment
0
Thanks for reporting your issue in details. Can you post the code you used to transfer? Does your server support file checksum?
 
answered 5/22/2018 4:16:46 AM
  I do the upload with uploadfile(localFile, RemoteFile) client.UploadFile(cFileBYesUpload, cFileIBMUpload) As I wrote, more than 41'000 files were uploaded without problems (so it's no problem with the code). I further don't know, if the sFTP server supports checksum (as I don't have any support from the server side). Please just answer my questions: - is this a known problem (sporadically corrupt files)? - can it be, that the sFTP library start to transfer a file, although the file is not written completely by the export application? - can it be, that the sFTP library cannot write the file completely on the sFTP server, as the file is opened on the sFTP Server, before it is written completely (whereby, I think, this should cause a crash of the transfer...?) - do you have some other ideas, what can cause the problems? Thanks fredy.wenger@matso.ch 5/22/2018 7:11:43 AM
add a comment

Your Answer

Not the answer you're looking for? Browse other questions tagged ultimate sftp or ask your own question.