What is the problem you are having with rclone?
Hi,
I sync all my files to Chunker-Crypt-Jottacloud which works very well. Instead of overwriting/deleting modified/deleted source files, I want to move them to a backup dir which I more or less regularly check and cleanup. This does not work as expected as files are not server-side moved to the backup dir but re-uploaded. I don't care for small files, but for big files it is annoying especially because this often fails and starts over and over again.
I'm afraid that there is nothing I can do about it as moving a file on the crypt remote doesn't mean that it can also be just moved on the "real" remote as the encrypted file is not the same before and after the move. Am I right in this assumption? Are there any workarounds that I cloud try?
Run the command 'rclone version' and share the full output of the command.
rclone v1.57.0
- os/version: debian 11.2 (64 bit)
- os/kernel: 5.10.0-10-amd64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.17.2
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
Jottacloud (with first chunker and then crypt)
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone -vv --delete-excluded --backup-dir=Chunker-Crypt-Jotta:/daten_backup/backup/2022-01-20_164504 --exclude=.sync/Archive/** sync /daten/backup Chunker-Crypt-Jotta:/daten/backup
The rclone config contents with secrets removed.
[Jotta]
type = jottacloud
configVersion = 1
client_id = jottacli
client_secret =
tokenURL = https://id.jottacloud.com/auth/realms/jottacloud/protocol/openid-connect/token
token = xxx
device =
mountpoint =
hard_delete = false
no_versions = true
[Crypt-Jotta]
type = crypt
remote = Jotta:_Backup/rclone_crypt
password = xxx
password2 = xxx
[Chunker-Crypt-Jotta]
type = chunker
remote = Crypt-Jotta:
chunk_size = 512Mi
hash_type = none
name_format = *.rc##
A log from the command with the -vv flag
2022/01/20 16:45:06 DEBUG : rclone: Version "v1.57.0" starting with parameters ["rclone" "-vv" "--delete-excluded" "--backup-dir=Chunker-Crypt-Jotta:/daten_backup/backup/2022-01-20_164504" "--exclude=.sync/Archive/**" "sync" "/daten/backup" "Chunker-Crypt-Jotta:/daten/backup"]
2022/01/20 16:45:06 DEBUG : Creating backend with remote "/daten/backup"
2022/01/20 16:45:06 DEBUG : Using config file from "/datenhd_luks/rclone.conf"
2022/01/20 16:45:06 DEBUG : Creating backend with remote "Chunker-Crypt-Jotta:/daten/backup"
2022/01/20 16:45:06 DEBUG : Creating backend with remote "Crypt-Jotta:/daten/backup"
2022/01/20 16:45:06 DEBUG : Creating backend with remote "Jotta:_Backup/rclone_crypt/6fjn533fam0j33igmppc8f9bj4/bl29jng1ptsj6j9aep2kek9o0c"
2022/01/20 16:45:06 DEBUG : jottacloud root '_Backup/rclone_crypt/6fjn533fam0j33igmppc8f9bj4/bl29jng1ptsj6j9aep2kek9o0c': Token expired but no uploads in progress - doing nothing
2022/01/20 16:45:06 DEBUG : Jotta: Loaded invalid token from config file - ignoring
2022/01/20 16:45:06 DEBUG : Saving config "token" in section "Jotta" of the config file
2022/01/20 16:45:06 DEBUG : Jotta: Saved new token in config file
2022/01/20 16:45:07 DEBUG : Reset feature "ListR"
2022/01/20 16:45:07 DEBUG : Creating backend with remote "Chunker-Crypt-Jotta:/daten_backup/backup/2022-01-20_164504"
2022/01/20 16:45:07 DEBUG : Creating backend with remote "Crypt-Jotta:/daten_backup/backup/2022-01-20_164504"
2022/01/20 16:45:07 DEBUG : Creating backend with remote "Jotta:_Backup/rclone_crypt/sm6oclu0un2as35drelteaogdg/bl29jng1ptsj6j9aep2kek9o0c/pqjg1i2pdvfei5tej1cllj09kbd0cumtd8bfcrvvcl3m5cs97rl0"
2022/01/20 16:45:07 DEBUG : Reset feature "ListR"
2022/01/20 16:45:07 NOTICE: _external_: Can't follow symlink without -L/--copy-links
(many skipped files)
2022/01/20 16:45:08 DEBUG : Chunked 'Chunker-Crypt-Jotta:/daten/backup': Waiting for transfers to finish
2022/01/20 16:45:08 DEBUG : Waiting for deletions to finish
2022/01/20 16:45:08 DEBUG : VM/Win10_Backblaze_2021-09-11.7z: Can't move - required hash not found
2022/01/20 16:45:08 DEBUG : VM/Win10_Backblaze_2021-09-11.7z: Can't move, switching to copy
2022/01/20 16:45:08 DEBUG : VM/Win10_Backblaze_2021-09-11.7z: Can't copy - required hash not found
2022/01/20 16:46:07 INFO :
Transferred: 39.375 MiB / 5.591 GiB, 1%, 1.203 MiB/s, ETA 1h18m47s
Checks: 440 / 441, 100%
Deleted: 1 (files), 0 (dirs)
Transferred: 0 / 1, 0%
Elapsed time: 1m1.6s
Checking:
Transferring:
* VM/Win10_Backblaze_2021-09-11.7z: 0% /5.591Gi, 1.214Mi/s, 1h18m5s
(some minutes later)
2022/01/20 17:58:07 INFO :
Transferred: 4.987 GiB / 5.591 GiB, 89%, 1.539 MiB/s, ETA 6m42s
Checks: 440 / 441, 100%
Deleted: 1 (files), 0 (dirs)
Transferred: 0 / 1, 0%
Elapsed time: 1h13m1.6s
Checking:
Transferring:
* VM/Win10_Backblaze_2021-09-11.7z: 89% /5.591Gi, 1.549Mi/s, 6m39s
2022/01/20 17:58:19 DEBUG : VM/Win10_Backblaze_2021-09-11.7z: md5 = 85c9ef31fac3fc4b330d58e4d83454f7 OK
2022/01/20 17:58:19 DEBUG : VM/Win10_Backblaze_2021-09-11.7z: Reopening on read failure after 5385486336 bytes: retry 10/10: read tcp 192.168.0.100:52688->185.179.130.30:443: i/o timeout
2022/01/20 17:58:19 DEBUG : VM/Win10_Backblaze_2021-09-11.7z: Reopen failed after 5385486336 bytes read: failed to reopen: too many retries
2022/01/20 17:58:21 DEBUG : VM/Win10_Backblaze_2021-09-11.7z: Received error: failed to calculate MD5: read tcp 192.168.0.100:52688->185.179.130.30:443: i/o timeout - low level retry 1/10
2022/01/20 17:58:21 DEBUG : VM/Win10_Backblaze_2021-09-11.7z: Can't copy - required hash not found
2022/01/20 17:59:07 INFO :
Transferred: 5.018 GiB / 10.592 GiB, 47%, 870.968 KiB/s, ETA 1h51m50s
Checks: 440 / 441, 100%
Deleted: 1 (files), 0 (dirs)
Transferred: 0 / 1, 0%
Elapsed time: 1h14m1.6s
Checking:
Transferring:
* VM/Win10_Backblaze_2021-09-11.7z: 0% /5.591Gi, 804.635Ki/s, 2h1m4s