What is the problem you are having with rclone?
Sync started through remote control abruptly stops with context canceled
errors
What is your rclone version (output from rclone version
)
I'm using the docker container
rclone v1.52.3
- os/arch: linux/arm64
- go version: go1.14.7
Which OS you are using and how many bits (eg Windows 7, 64 bit)
The host OS is Ubuntu Server, 64-bit on a Raspberry Pi.
cobertos@file-bean:~/batto-cave-backup$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.1 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.1 LTS"
Which cloud storage system are you using? (eg Google Drive)
Seafile
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
To my Docker container
rcd --rc-web-gui --rc-web-gui-no-open-browser --rc-addr=0.0.0.0:5572 --rc-user=test --rc-pass=test --rc-enable-metrics
Inside the Docker container after starting
rclone rc --rc-user=test --rc-pass=test -vv sync/sync srcFs=battoseafile: dstFs=/backup-dest
The rclone config contents with secrets removed.
Not using a config, here's my docker-compose.yml
version: "3.6"
services:
rclone:
image: rclone/rclone
container_name: rclone
restart: always
environment:
# Configuring without a configuration file (using ENV variables with special
# naming)
# https://rclone.org/docs/#config-file
# https://rclone.org/seafile/
RCLONE_CONFIG_BATTOSEAFILE_TYPE: seafile
RCLONE_CONFIG_BATTOSEAFILE_URL: https://seafile.xxx.xxx
RCLONE_CONFIG_BATTOSEAFILE_USER: xxx@gmail.com
# This needs to be the output of `docker container run rclone/rclone obscure [PASSWORD]`
RCLONE_CONFIG_BATTOSEAFILE_PASS: $SEAFILE_OBSCURED_PASSWORD
command:
- "rcd"
- "--rc-web-gui"
- "--rc-web-gui-no-open-browser"
- "--rc-addr=0.0.0.0:5572"
- "--rc-user=test"
- "--rc-pass=test"
- "--rc-enable-metrics" # For prometheus metrics
ports:
- "5572:5572"
volumes:
- /mnt/ssd/batto-backup:/backup-dest
A log from the command with the -vv
flag
From the Docker container itself (the rcd command), this is just the tail.
...
rclone | 2020/08/28 11:32:09 INFO : pics/NewRL/OldRLToSort/IMG_2515.JPG: Copied (new)
rclone | 2020/08/28 11:32:09 INFO : pics/NewRL/OldRLToSort/IMG_2516.JPG: Copied (new)
rclone | 2020/08/28 11:32:10 INFO : pics/NewRL/OldRLToSort/IMG_2517.JPG: Copied (new)
rclone | 2020/08/28 11:32:10 INFO : pics/NewRL/OldRLToSort/IMG_2518.JPG: Copied (new)
rclone | 2020/08/28 11:32:10 INFO : pics/NewRL/OldRLToSort/IMG_2519.JPG: Copied (new)
rclone | 2020/08/28 11:32:10 INFO : pics/NewRL/OldRLToSort/IMG_2520.JPG: Copied (new)
rclone | 2020/08/28 11:32:10 INFO : pics/NewRL/OldRLToSort/IMG_2521.JPG: Copied (new)
rclone | 2020/08/28 11:32:10 INFO : pics/NewRL/OldRLToSort/IMG_2522.JPG: Copied (new)
rclone | 2020/08/28 11:32:11 DEBUG : rc: "core/stats": with parameters map[]
rclone | 2020/08/28 11:32:11 DEBUG : rc: "core/stats": with parameters map[]
rclone | 2020/08/28 11:32:11 DEBUG : rc: "core/stats": reply map[bytes:2773389499 checks:9979 deletes:0 elapsedTime:23.386264832 errors:0 fatalError:false renames:0 retryError:false speed:1.1859129963087212e+08 transferring:[map[name:pics/NewRL/OldRLToSort/IMG_2526.JPG size:1341845] map[name:pics/NewRL/OldRLToSort/IMG_2523.JPG size:1166306] map[name:pics/NewRL/OldRLToSort/IMG_2524.JPG size:1229875] map[name:pics/NewRL/OldRLToSort/IMG_2525.JPG size:969565]] transfers:1048]: <nil>
rclone | 2020/08/28 11:32:11 DEBUG : rc: "core/stats": reply map[bytes:2773389499 checks:9979 deletes:0 elapsedTime:23.386443292 errors:0 fatalError:false renames:0 retryError:false speed:1.1859028986371039e+08 transferring:[map[name:pics/NewRL/OldRLToSort/IMG_2526.JPG size:1341845] map[name:pics/NewRL/OldRLToSort/IMG_2523.JPG size:1166306] map[name:pics/NewRL/OldRLToSort/IMG_2524.JPG size:1229875] map[name:pics/NewRL/OldRLToSort/IMG_2525.JPG size:969565]] transfers:1048]: <nil>
rclone | 2020/08/28 11:32:11 INFO : pics/NewRL/OldRLToSort/IMG_2523.JPG: Copied (new)
rclone | 2020/08/28 11:32:11 INFO : pics/NewRL/OldRLToSort/IMG_2524.JPG: Copied (new)
rclone | 2020/08/28 11:32:11 INFO : pics/NewRL/OldRLToSort/IMG_2525.JPG: Copied (new)
rclone | 2020/08/28 11:32:11 INFO : pics/NewRL/OldRLToSort/IMG_2526.JPG: Copied (new)
rclone | 2020/08/28 11:32:12 INFO : pics/NewRL/OldRLToSort/IMG_2527.JPG: Copied (new)
rclone | 2020/08/28 11:32:12 INFO : pics/NewRL/OldRLToSort/IMG_2528.JPG: Copied (new)
rclone | 2020/08/28 11:32:12 INFO : pics/NewRL/OldRLToSort/IMG_2529.JPG: Copied (new)
rclone | 2020/08/28 11:32:12 INFO : pics/NewRL/OldRLToSort/IMG_2530.JPG: Copied (new)
rclone | 2020/08/28 11:32:13 INFO : pics/NewRL/OldRLToSort/IMG_2531.JPG: Copied (new)
rclone | 2020/08/28 11:32:13 INFO : pics/NewRL/OldRLToSort/IMG_2532.JPG: Copied (new)
rclone | 2020/08/28 11:32:13 INFO : pics/NewRL/OldRLToSort/IMG_2533.JPG: Copied (new)
rclone | 2020/08/28 11:32:13 INFO : pics/NewRL/OldRLToSort/IMG_2534.JPG: Copied (new)
rclone | 2020/08/28 11:32:14 INFO : pics/NewRL/OldRLToSort/IMG_2535.JPG: Copied (new)
rclone | 2020/08/28 11:32:14 INFO : pics/NewRL/OldRLToSort/IMG_2541.JPG: Copied (new)
rclone | 2020/08/28 11:32:14 INFO : pics/NewRL/OldRLToSort/IMG_2542.JPG: Copied (new)
rclone | 2020/08/28 11:32:14 INFO : pics/NewRL/OldRLToSort/IMG_2543.JPG: Copied (new)
rclone | 2020/08/28 11:32:14 INFO : pics/NewRL/OldRLToSort/IMG_2633.JPG: Copied (new)
rclone | 2020/08/28 11:32:15 INFO : pics/NewRL/OldRLToSort/IMG_2638.JPG: Copied (new)
rclone | 2020/08/28 11:32:15 INFO : pics/NewRL/OldRLToSort/IMG_2639.JPG: Copied (new)
rclone | 2020/08/28 11:32:15 INFO : pics/NewRL/OldRLToSort/IMG_2642.JPG: Copied (new)
rclone | 2020/08/28 11:32:15 ERROR : archive/OldDev/Arduino/libraries/U8glib/src/clib: error reading source directory: failed to get directory contents: Get "https://seafile.xxx.xxx/api/v2.1/repos/7ad981b6-dc7d-47e0-bd0a-b02133924b45/dir/?p=%2FOldDev%2FArduino%2Flibraries%2FU8glib%2Fsrc%2Fclib&recursive=0": context canceled
rclone | 2020/08/28 11:32:15 ERROR : pics/NewRL/OldRLToSort/IMG_2650.JPG: Failed to copy: failed to open source object: failed to get download link: Get "https://seafile.xxx.xxx/api2/repos/3b448165-d541-403e-a3c4-9127d2712f74/file/?p=%2FNewRL%2FOldRLToSort%2FIMG_2650.JPG": context canceled
rclone | 2020/08/28 11:32:15 ERROR : pics/NewRL/OldRLToSort/IMG_2643.JPG: Failed to copy: failed to open source object: Get "https://seafile.xxx.xxx/seafhttp/files/1a687204-c474-4613-ad22-7d3a705b6d96/IMG_2643.JPG": context canceled
rclone | 2020/08/28 11:32:15 ERROR : pics/NewRL/OldRLToSort/IMG_2645.JPG: Failed to copy: failed to open source object: Get "https://seafile.xxx.xxx/seafhttp/files/dea9ce5c-ba61-42e1-b353-d51e424ff4d8/IMG_2645.JPG": context canceled
rclone | 2020/08/28 11:32:15 ERROR : pics/NewRL/OldRLToSort/IMG_2649.JPG: Failed to copy: failed to open source object: Get "https://seafile.xxx.xxx/seafhttp/files/90345f7f-56b8-4391-ae05-0e1c786a29a8/IMG_2649.JPG": context canceled
rclone | 2020/08/28 11:32:16 ERROR : games/RollerCoaster Tycoon® 3: error reading source directory: failed to get directory contents: Get "https://seafile.xxx.xxx/api/v2.1/repos/ac77d399-77be-495b-93bc-97b2295869bc/dir/?p=%2FRollerCoaster+Tycoon%C2%AE+3&recursive=0": context canceled
rclone | 2020/08/28 11:32:16 DEBUG : Local file system at /backup-dest: Waiting for checks to finish
rclone | 2020/08/28 11:32:16 DEBUG : Local file system at /backup-dest: Waiting for transfers to finish
rclone | 2020/08/28 11:32:16 ERROR : Local file system at /backup-dest: not deleting files as there were IO errors
rclone | 2020/08/28 11:32:16 ERROR : Local file system at /backup-dest: not deleting directories as there were IO errors
rclone | 2020/08/28 11:32:16 ERROR : rc: "sync/sync": error: context canceled
rclone | 2020/08/28 11:32:16 DEBUG : rc: "core/stats": with parameters map[]
rclone | 2020/08/28 11:32:16 DEBUG : rc: "core/stats": with parameters map[]
rclone | 2020/08/28 11:32:16 DEBUG : rc: "core/stats": reply map[bytes:2797834055 checks:9979 deletes:0 elapsedTime:22.886622409 errors:6 fatalError:false lastError:failed to get directory contents: Get "https://seafile.xxx.xxx/api/v2.1/repos/ac77d399-77be-495b-93bc-97b2295869bc/dir/?p=%2FRollerCoaster+Tycoon%C2%AE+3&recursive=0": context canceled renames:0 retryError:true speed:1.2224757349515112e+08 transfers:1068]: <nil>
rclone | 2020/08/28 11:32:16 DEBUG : rc: "core/stats": reply map[bytes:2797834055 checks:9979 deletes:0 elapsedTime:22.886622409 errors:6 fatalError:false lastError:failed to get directory contents: Get "https://seafile.xxx.xxx/api/v2.1/repos/ac77d399-77be-495b-93bc-97b2295869bc/dir/?p=%2FRollerCoaster+Tycoon%C2%AE+3&recursive=0": context canceled renames:0 retryError:true speed:1.2224757349515112e+08 transfers:1068]: <nil>
rclone | 2020/08/28 11:32:21 DEBUG : rc: "core/stats": with parameters map[]
rclone | 2020/08/28 11:32:21 DEBUG : rc: "core/stats": reply map[bytes:2797834055 checks:9979 deletes:0 elapsedTime:22.886622409 errors:6 fatalError:false lastError:failed to get directory contents: Get "https://seafile.xxx.xxx/api/v2.1/repos/ac77d399-77be-495b-93bc-97b2295869bc/dir/?p=%2FRollerCoaster+Tycoon%C2%AE+3&recursive=0": context canceled renames:0 retryError:true speed:1.2224757349515112e+08 transfers:1068]: <nil>
rclone | 2020/08/28 11:32:21 DEBUG : rc: "core/stats": with parameters map[]
rclone | 2020/08/28 11:32:21 DEBUG : rc: "core/stats": reply map[bytes:2797834055 checks:9979 deletes:0 elapsedTime:22.886622409 errors:6 fatalError:false lastError:failed to get directory contents: Get "https://seafile.xxx.xxx/api/v2.1/repos/ac77d399-77be-495b-93bc-97b2295869bc/dir/?p=%2FRollerCoaster+Tycoo
Output from rclone rc ... sync/sync
:
2020/08/28 11:27:15 DEBUG : rclone: Version "v1.52.3" starting with parameters ["rclone" "rc" "--rc-user=test" "--rc-pass=test" "-vv" "sync/sync" "srcFs=battoseafile:" "dstFs=/backup-dest"]
2020/08/28 11:32:15 DEBUG : 4 go routines active
2020/08/28 11:32:15 Failed to rc: connection failed: Post "http://localhost:5572/sync/sync": net/http: timeout awaiting response headers
Interesting, I was not able to reproduce this with the direct rclone command:
rclone sync -vv battoseafile: /backup-dest
I'm going to let it run the rest of the night with this direct command and will report if there are any of the same issues.