Exit Codes and partial transfers

What is the problem you are having with rclone?

It it correct that if rclone has a large file transfer or multipart upload transfer interrupted, and then runs again, it will check those files (using dates or checksum depending on command flags), and retry the upload if the pieces or file are incomplete?
As part of that, if the re-upload fails, rclone will exit with a non-0 exit code?
(I'm making a script to automate uploads and I want to make sure that exit 0 is definitely "everything is uploaded and command completed successfully")

What is your rclone version (output from rclone version)

1.52.2

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Win Server 2016

Which cloud storage system are you using? (eg Google Drive)

Amazon S3 (AWS)

hello,

  • rclone uploads the entire file, not pieces of a file. if a back-end supports it, then rclone can use multipart to upload the entire file.
    if an upload fails, next time rclone is run, it will upload the entire file.

  • https://rclone.org/commands/rclone_copy/
    "Copy the source to the destination. Doesn't transfer unchanged files, testing by size and modification time"

  • if rclone fails to upload a file, it will generate a non-zero error code.
    https://rclone.org/docs/#list-of-exit-codes
    you can test this by disabling your network connection and let rclone fail.

1 Like

Rclone does multipart uploads for large files in chunks to S3 doesn't it?

yes,
https://rclone.org/s3/#multipart-uploads

if rclone is doing a multi-part upload of 100 parts
and rclone has upload 99 parts and then fails to upload the last part.
next time rclone is run, it will have to upload 100 parts, not just the last part.