RClone Not Finishing Broken Transfers

Suspected Bug
1

What is the problem you are having with rclone?

After an interrupted transfer is resumed, files that were partially transferred are not resumed. Transfers continue at the next file.

Run the command 'rclone version' and share the full output of the command.

rclone v1.57.0

  • os/version: ubuntu 20.04 (64 bit)
  • os/kernel: 5.4.0-100-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.17.2
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

SFTP on LAN

The command you were trying to run (eg rclone copy /tmp remote:tmp)

"rclone" "--password-command" "base64 -d /home/david/rcp.64" "--check-first" "--transfers" "10" "-vv" "copy" "/mnt/easystore/plexbkups" "QNap2:Backups/plexbkups"

NOTE: that remote is called "QNap" because of the hardware, but it's running Debian 11.

The rclone config contents with secrets removed.

    "QNap2": {
        "host": "frier-nas-02.home.arpa",
        "key_file": "~/.ssh/david@plex",
        "md5sum_command": "md5sum",
        "sha1sum_command": "sha1sum",
        "type": "sftp",
        "use_insecure_cipher": "false"
    }

A log from the command with the -vv flag

https://gist.github.com/kahomono/f403a33212e82e30fb85fed43fd3e5b1
2

There's isn't a resume upload. There are a few features to enable that, but it doesn't exist yet.

Resume uploads · Issue #87 · rclone/rclone (github.com)

I'm sure there a few more as well.

3

Is there, in that case, a way to ignore timestamp and force (re-) sending any files that are different in size or checksum?

4

Rclone will upload a file that differs already.

Pick a single file and test and validate it's working to your needs.

5

It's only uploading different files if the source's timestamp is later than the destination's.

In this case, rclone was ^C halted while uploading a 200M file with a Feb 14 03:00 timestamp (example, but this is what I have observed multiple times). The file on the destination was 171M with a timestamp of the moment ^C got hit. rclone skipped that file when started again, because (I presume) the destination timestamp was later than the source.

6

Can you share the specific file / log file with debug -vv?

7

this is what I have already. Is it possible to get more?

8

^^^^

I can't go through a huge log and guess which one file works or doesn't work. Can you please pick one file that breaks the process and share that log?

9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.