Sync command seems to loop

I am trying to sync two locations with a large number of small files and it takes forever.
The first sync took 4 days and finished with some warnings.
The second run to check if something was missing (no actual copy) after 24h it hadn't finished and in the log file it showed the same NOTICE entries after 6-8 hours (it could be retries).

The source contains 9.512.000 files and 4.589.000 folders.
The sync reports (below) 23million checks..

rclone sync G:\ H:\ --progress --log-file Elements-Maxtor-sync01.log --log-level INFO
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors: 1 (retrying may help)
Checks: 23068554 / 23068554, 100%
Transferred: 0 / 0, -
Elapsed time: 24h16m42.2s

rclone --version
rclone v1.47.0

  • os/arch: windows/amd64
  • go version: go1.11

any idea??

Are G: and H: local mounts or what are they?

yes they are local but I want afterwards to use the second one as a backup of a network folder (same files)

Is there a reason to use rclone as another sync tool for local files would be better imo, like rsync.

I have a windows environment and wanted to avoid robocopy..

what does

rclone size G:\
rclone size H:\

return? Seems odd you have 23M checks but way less files. Are there any special junctions or sym links in those file systems?

After 24+ hours of not finishing (rclone size) I stopped it..
The check with robocopy (the actual copy was done by rclone) has run now more than 26h and still has not finished..

Run with debug. -vv.

rclone size H:\ -vv
2019/06/11 10:32:55 DEBUG : rclone: Version "v1.47.0" starting with parameters ["C:\SW\Chocolatey\lib\rclone.portable\tools\rclone-v1.47.0-windo
ws-amd64\rclone.exe" "size" "H:\" "-vv"]
2019/06/11 10:32:55 NOTICE: Config file "C:\Users\xxx\.config\rclone\rclone.conf" not found - using defaults
2019/06/11 10:32:55 ERROR : System Volume Information: error listing: failed to open directory "System Volume Information": open \?\H:\System Volume
Information: Access is denied.
2019/06/11 12:34:37 Failed to size: failed to open directory "System Volume Information": open \?\H:\System Volume Information: Access is denied.

it is run in an elevated powershell.. (similar for the other drive)

You can put a filter in to ignore that directory --exclude "/System Volume Information/**" which might help.