What is the problem you are having with rclone?
Rclone sync of Swift container consisting of docker files taking long time. The below command is set to sync data for last 7 hrs on Monday evening and its been running since then. The sync is happening between US Data Center and data center in Europe.
The initial sync of all the data took around 2 weeks. I believe its only the delta (daily code updates will not be more that few GBs). Appreciate if someone can help with the right flags to speed up the sync.
What is your rclone version (output from
- os/arch: linux/amd64
- go version: go1.15
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Which cloud storage system are you using? (eg Google Drive)
The command you were trying to run (eg
rclone copy /tmp remote:tmp)
rclone sync Swift1:registry Swift2:dockerregistry --fast-list --checksum --ignore-existing --max-age 7h --log-file /var/log/rclone/rclonesync.log --log-level DEBUG
The rclone config contents with secrets removed.
[Swift1] type = swift user = repo key = ******** auth = http://url/auth/v1.0 domain = tenant = tenant_domain = region = storage_url = auth_version = 1 [Swift2] type = swift user = svc.prod key = ****** auth = http://url/auth/v1.0 domain = tenant = tenant_domain = region = storage_url = auth_version = 1
A log from the command with the
2020/10/08 21:02:09 DEBUG : images/files/docker/v2/repositories/copart/ycsdmvsmus/_layers/sha256/eff7a10a26da26c34360c78b6dff31cb179c2cdad454tre4yg/link: Excluded from sync (and deletion) 2020/10/08 21:02:09 DEBUG : images/files/docker/v2/repositories/copart/ycsdmvsmus/_layers/sha256/f0fd78c1cca8d49323a5910a6b7a7a5ac541af721ee4188f23345544fe/link: Excluded from sync (and deletion)