Rclone sftp resume transfer on network loss

Help and Support
1

What is the problem you are having with rclone?

if there is connection break during active rclone sync, and rclone is restarted, it doesnt resume from previous point

What is your rclone version (output from rclone version)

rclone v1.52.3

  • os/arch: linux/amd64
  • go version: go1.14.7

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Centos 7.6 x64

Which cloud storage system are you using? (eg Google Drive)

sftp

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync sg1:/home/qb/tmp/2020.08.18.tar.gz .

The rclone config contents with secrets removed.

[sg1]
type = sftp
host = qbcsg1
user = qb
key_file = /home/qb/.ssh/id_rsa
md5sum_command = md5sum
disable_hashcheck = true
sha1sum_command = sha1sum

Rclone does not resume file transfer from previous point, ie if it synced 400mb out of 700mb and breaks at 400, when resumed, it starts again at 0mb and starts syncing.

how can I make it resume the sync?

2

It does.

The last line you deleted from the template which is the log file would show that.

If you want to post the logfile, that would be great and we can look at it.

3

I ran it with vvv, its picking up a cache somewhere, but its always starting at around 360 mb, even if I sync to lets say 500mb, stop the sync, and restart, the cache is always restarting at 360mb

sclone sync sg1:/home/qb/tmp/2020.08.18.tar.gz . -vvvv
2020/08/28 15:02:00 DEBUG : rclone: Version "v1.52.3" starting with parameters ["sclone" "sync" "sg1:/home/qb/tmp/2020.08.18.tar.gz" "." "-vvvv"]
2020/08/28 15:02:00 DEBUG : Using config file from "/home/qb/.config/rclone/rclone.conf"
2020/08/28 15:02:02 DEBUG : sftp://qb@qbcsg1:22//home/qb/tmp/2020.08.18.tar.gz: New connection 192.168.37.8:58814->10.251.1.18:22 to "SSH-2.0-OpenSSH_7.4"
2020/08/28 15:02:04 DEBUG : fs cache: adding new entry for parent of "sg1:/home/qb/tmp/2020.08.18.tar.gz", "sg1:/home/qb/tmp"
2020/08/28 15:02:04 DEBUG : fs cache: renaming cache item "." to be canonical "/home/qb"
2020/08/28 15:02:04 DEBUG : 2020.08.18.tar.gz: Sizes differ (src 755333865 vs dst 378404864)
2020/08/28 15:02:04 DEBUG : 2020.08.18.tar.gz: Starting multi-thread copy with 2 parts of size 360.188M
2020/08/28 15:02:04 DEBUG : 2020.08.18.tar.gz: multi-thread copy: stream 1/2 (0-377683968) size 360.188M starting
2020/08/28 15:02:04 DEBUG : 2020.08.18.tar.gz: multi-thread copy: stream 2/2 (377683968-755333865) size 360.155M starting
2020/08/28 15:02:06 DEBUG : sftp://qb@qbcsg1:22//home/qb/tmp/2020.08.18.tar.gz: New connection 192.168.37.8:58846->10.251.1.18:22 to "SSH-2.0-OpenSSH_7.4"
4

ignore the "sclone" thats an alias for rclone, we had another alias in bashrc for "rclone" that git clones an R-lang repository, so I had to create a new alias

5

There isn't not a resume upload option.

If you sync say 100 files and it stops at 50, the remaining 50 would resume.

See:

6

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.