Rclone union with one host down

What is the problem you are having with rclone?

I am creating multiple archive servers. If a server goes down I want rclone to keep working with the remaining ones.

I have two remotes for now ryzen7: and zuper: for now. I stopped the webdav server on zuper: to test what would happen if one of the hosts would become unavailable.

What is your rclone version (output from rclone version)

docker run --rm -it -v ~/.config/rclone/rclone.conf:/config/rclone/rclone.conf -v ~/chiaplotter:/costi rclone/rclone --version                                22:07:39
rclone v1.54.0
- os/arch: linux/amd64
- go version: go1.15.7

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Linux 64 bit

Which cloud storage system are you using? (eg Google Drive)

Webdav with rclone serve webdav

The command you were trying to run (eg rclone copy /tmp remote:tmp)

docker run --rm -it -v ~/.config/rclone/rclone.conf:/config/rclone/rclone.conf -v ~/chiaplotter:/costi rclone/rclone copy /costi/Dockerfile archives:/Dockerfile

2021/03/16 03:06:20 Failed to create file system for "archives:/Dockerfile": read metadata failed: Propfind "http://costi-linux-zuper:8080/Dockerfile": dial tcp 192.168.2.145:8080: connect: connection refused

The rclone config contents with secrets removed.

cat ~/.config/rclone/rclone.conf                                                                                                                              22:07:46
[ryzen7]
type = webdav
url = http://ryzen7.home:8080
vendor = other

[zuper]
type = webdav
url = http://costi-linux-zuper:8080
vendor = other

[archives]
type = union
upstreams = ryzen7: zuper:

A log from the command with the -vv flag

docker run --rm -it -v ~/.config/rclone/rclone.conf:/config/rclone/rclone.conf -v ~/chiaplotter:/costi rclone/rclone copy /costi/Dockerfile archives:/Dockerfile -vv
2021/03/16 03:10:22 DEBUG : rclone: Version "v1.54.0" starting with parameters ["rclone" "copy" "/costi/Dockerfile" "archives:/Dockerfile" "-vv"]
2021/03/16 03:10:22 DEBUG : Creating backend with remote "/costi/Dockerfile"
2021/03/16 03:10:22 DEBUG : Using config file from "/config/rclone/rclone.conf"
2021/03/16 03:10:22 DEBUG : fs cache: adding new entry for parent of "/costi/Dockerfile", "/costi"
2021/03/16 03:10:22 DEBUG : Creating backend with remote "archives:/Dockerfile"
2021/03/16 03:10:22 DEBUG : Creating backend with remote "zuper:"
2021/03/16 03:10:22 DEBUG : Creating backend with remote "ryzen7:"
2021/03/16 03:10:22 DEBUG : Creating backend with remote "zuper:/Dockerfile"
2021/03/16 03:10:22 DEBUG : Creating backend with remote "ryzen7:/Dockerfile"
2021/03/16 03:10:22 DEBUG : pacer: low level retry 1/10 (error Propfind "http://costi-linux-zuper:8080/Dockerfile": dial tcp 192.168.2.145:8080: connect: connection refused)
2021/03/16 03:10:22 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2021/03/16 03:10:22 DEBUG : fs cache: renaming cache item "ryzen7:/Dockerfile" to be canonical "ryzen7:Dockerfile"
2021/03/16 03:10:22 DEBUG : pacer: low level retry 2/10 (error Propfind "http://costi-linux-zuper:8080/Dockerfile": dial tcp 192.168.2.145:8080: connect: connection refused)
2021/03/16 03:10:22 DEBUG : pacer: Rate limited, increasing sleep to 40ms
2021/03/16 03:10:22 DEBUG : pacer: low level retry 3/10 (error Propfind "http://costi-linux-zuper:8080/Dockerfile": dial tcp 192.168.2.145:8080: connect: connection refused)
2021/03/16 03:10:22 DEBUG : pacer: Rate limited, increasing sleep to 80ms
2021/03/16 03:10:22 DEBUG : pacer: low level retry 4/10 (error Propfind "http://costi-linux-zuper:8080/Dockerfile": dial tcp 192.168.2.145:8080: connect: connection refused)
2021/03/16 03:10:22 DEBUG : pacer: Rate limited, increasing sleep to 160ms
2021/03/16 03:10:22 DEBUG : pacer: low level retry 5/10 (error Propfind "http://costi-linux-zuper:8080/Dockerfile": dial tcp 192.168.2.145:8080: connect: connection refused)
2021/03/16 03:10:22 DEBUG : pacer: Rate limited, increasing sleep to 320ms
2021/03/16 03:10:23 DEBUG : pacer: low level retry 6/10 (error Propfind "http://costi-linux-zuper:8080/Dockerfile": dial tcp 192.168.2.145:8080: connect: connection refused)
2021/03/16 03:10:23 DEBUG : pacer: Rate limited, increasing sleep to 640ms
2021/03/16 03:10:23 DEBUG : pacer: low level retry 7/10 (error Propfind "http://costi-linux-zuper:8080/Dockerfile": dial tcp 192.168.2.145:8080: connect: connection refused)
2021/03/16 03:10:23 DEBUG : pacer: Rate limited, increasing sleep to 1.28s
2021/03/16 03:10:23 DEBUG : pacer: low level retry 8/10 (error Propfind "http://costi-linux-zuper:8080/Dockerfile": dial tcp 192.168.2.145:8080: connect: connection refused)
2021/03/16 03:10:23 DEBUG : pacer: Rate limited, increasing sleep to 2s
2021/03/16 03:10:25 DEBUG : pacer: low level retry 9/10 (error Propfind "http://costi-linux-zuper:8080/Dockerfile": dial tcp 192.168.2.145:8080: connect: connection refused)
2021/03/16 03:10:27 DEBUG : pacer: low level retry 10/10 (error Propfind "http://costi-linux-zuper:8080/Dockerfile": dial tcp 192.168.2.145:8080: connect: connection refused)
2021/03/16 03:10:27 Failed to create file system for "archives:/Dockerfile": read metadata failed: Propfind "http://costi-linux-zuper:8080/Dockerfile": dial tcp 192.168.2.145:8080: connect: connection refused

I thought there was an issue about this, but apparently not: Issues · rclone/rclone · GitHub

Do you want to create a new one about union resilience in the presence of upstream failure?

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.