Is it possible that a failed rclone copy results in a 0 byte output file?
Referring to https://github.com/cjnaz/rclonesync-V2/issues/56, there appear to be cases where if the transfer is terminated during a file copy that a 0 byte (or perhaps incomplete) file gets created at the destination, which is probably data loss for a bi-directional sync.
The transfer may fail for several reasons, including.. the internet connection drops, the client machine is hibernated, crashes or is rebooted.
Questions...
When a copy (or a sync) is in process, is a temporary file created at the destination, and renamed/moved to the final output name at the end of the successful transfer?
If so, are all remotes as destinations supported for this behavior?
Is a transfer to a local filesystem supported for this behavior?
Are there any rclone switches related to this behavior?
If this behavior is not supported, what are the blocking reasons?
What is your rclone version (output from rclone version)
rclone v1.52.2
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Centos 7.8 64b and Windows 10 64b
Which cloud storage system are you using? (eg Google Drive)
Google Drive, Local, Owncloud, ...
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone copy GDrive:fileXYZ <localdrivepath>
or
rclone copy <localdrivepath> GDrive:fileXYZ
However on local, sftp and ftp I think this probably is possible.
Rclone tries to tidy up failed copies by deleting partially transferred files. This doesn't always work if the user quits rclone before the end.
Normally this doesn't matter as rclone will just see the file is wrong the next time the transfer is re-run and retry however it looks like it does matter for your application.
From the issue it looks like it was a multipart copy that failed to the local disk.
Hmm when copying to local disk, ftp, etc then yes the destination file is overwritten at the start of the copy so if the copy fails the destination file will be corrupted.
Rsync has a mode where it transfers new files with a name like file.txt.xxxrandomstringxxx and renames it into place at the end.
This probably needs to be implemented on the local backend, so files are transferred with a temporary name (if there is an existing file of that name) then renamed into place at the end.
This could be a local backend option or it could be a global option for rclone which is implemented by the core copy routine.
For now I'll look into implementing temp file naming on local/sftp/ftp on Path1. I use an rclone sync Path1 Path2 as the final step, so would need rclone built in support if a local/sftp/ftp target is on Path2. I'll put in a feature request on github.
