Sync - file checks behavior question

rclone version
rclone v1.47.0

  • os/arch: linux/amd64
  • go version: go1.11.5

Source Server - CentOS 7
Destination Object Storage - DigitalOcean Spaces

If I'm counting correctly, the following local folder has approximately 1,960,000 files in it.
[root@hello ~]# find . /local/path/folder -type f|wc -l
[root@hello ~]# find . /local/path/folder -type f|wc -l

I stopped the sync job after 29 hours and what appears to be a little over 5.5 million file checks.
I know the digitalocean space was already very close to the same content as the source because I completed similar sync job 5 or 6 days earlier.

Why are there almost 3 times the number of file checks versus the total number of files at both the source and the destination?

[root@hello]# rclone sync /local/path/folder/ remote-name:bucket-name/2020/ -P -vv --bwlimit 1250 --delete-during --stats-log-level INFO --log-file=/var/log/rclone-sync-2020-05-12.txt
Transferred: 7.470G / 7.470 GBytes, 100%, 75.008 kBytes/s, ETA 0s
Errors: 3 (retrying may help)
Checks: 5537859 / 5547873, 100%
Transferred: 6524 / 6524, 100%
Elapsed time: 29h0m26.9s

Retries usually.

Your rclone version is pretty ancient too.

Thanks, I tried grepping the 1.8 GB log for "retry" but nothing found.
I'll take your word for it and possibly see about upgrading my rclone version.
Thanks for your help; still getting sorted using rclone vs s3cmd.

this will update rclone
curl | sudo bash

Holy crow that was supremely easy :grinning:

I don't believe you see retries in just an INFO log as they are not errors.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.