Do I need the `--download` flag for local-to-local checks?

Help and Support
1

Run the command 'rclone version' and share the full output of the command.

rclone v1.71.2
- os/version: Microsoft Windows 10 Pro 22H2 22H2 (64 bit)
- os/kernel: 10.0.19045.6456 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.25.3
- go/linking: static
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

I'm using only local-to-local data transfers, so it all happens on my PC.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone check D:\Music K:\D\Music --checkers 1 --download --progress --verbose

My question:

Do I need to add the --download flag if I'm checking local-to-local data transfers so that it really checks all the data and computes MD5 hashes? Or is it implied that you only download from cloud storage and that I don't need it for local transfers?

2

Run a test with debug logging (-vv flag) and see whether checksums are checked or not.

I think it should calculate hashes from local remotes without any extra flags - but to be sure I would test too.

3

i ran some tests, not sure what rclone is doing?
and there are a number of differences, in output, between the check commands?

note: my assumption, the elapsed time per command should indicate how rclone is handling checksums.

22.6s rclone check d:\test\check\src d:\test\check\dst
3.4s rclone check d:\test\check\src d:\test\check\dst --download

rclone copy   d:\test\check\src d:\test\check\dst -vv --inplace 
2025/10/27 14:45:19 DEBUG : rclone: Version "v1.71.0" starting with parameters ["d:\\data\\rclone\\rclone.exe" "copy" "d:\\test\\check\\src" "d:\\test\\check\\dst" "-vv" "--inplace"]
2025/10/27 14:45:19 DEBUG : Creating backend with remote "d:\\test\\check\\src"
2025/10/27 14:45:19 DEBUG : Using config file from "d:\\data\\rclone\\rclone.conf"
2025/10/27 14:45:19 DEBUG : fs cache: renaming cache item "d:\\test\\check\\src" to be canonical "//?/d:/test/check/src"
2025/10/27 14:45:19 DEBUG : Creating backend with remote "d:\\test\\check\\dst"
2025/10/27 14:45:19 DEBUG : fs cache: renaming cache item "d:\\test\\check\\dst" to be canonical "//?/d:/test/check/dst"
2025/10/27 14:45:19 DEBUG : file.ext: Need to transfer - File not found at Destination
2025/10/27 14:45:19 DEBUG : Local file system at //?/d:/test/check/dst: Waiting for checks to finish
2025/10/27 14:45:19 DEBUG : Local file system at //?/d:/test/check/dst: Waiting for transfers to finish
2025/10/27 14:45:44 DEBUG : file.ext: md5 = 52d37544ca98287dd2b15f526ebfafa5 OK
2025/10/27 14:45:44 INFO  : file.ext: Copied (new)
2025/10/27 14:45:44 INFO  : 
Transferred:   	    5.601 GiB / 5.601 GiB, 100%, 228.046 MiB/s, ETA 0s
Checks:                 0 / 0, -, Listed 1
Transferred:            1 / 1, 100%
Elapsed time:        25.1s
2025/10/27 14:45:44 DEBUG : 1 go routines active

-------------
rclone md5sum d:\test\check\src       -vv 
2025/10/27 14:45:47 DEBUG : rclone: Version "v1.71.0" starting with parameters ["d:\\data\\rclone\\rclone.exe" "md5sum" "d:\\test\\check\\src" "-vv"]
2025/10/27 14:45:47 DEBUG : Creating backend with remote "d:\\test\\check\\src"
2025/10/27 14:45:47 DEBUG : Using config file from "d:\\data\\rclone\\rclone.conf"
2025/10/27 14:45:47 DEBUG : fs cache: renaming cache item "d:\\test\\check\\src" to be canonical "//?/d:/test/check/src"
52d37544ca98287dd2b15f526ebfafa5  file.ext
2025/10/27 14:46:09 DEBUG : 2 go routines active

-------------
rclone md5sum d:\test\check\dst       -vv 
2025/10/27 14:46:12 DEBUG : rclone: Version "v1.71.0" starting with parameters ["d:\\data\\rclone\\rclone.exe" "md5sum" "d:\\test\\check\\dst" "-vv"]
2025/10/27 14:46:12 DEBUG : Creating backend with remote "d:\\test\\check\\dst"
2025/10/27 14:46:12 DEBUG : Using config file from "d:\\data\\rclone\\rclone.conf"
2025/10/27 14:46:12 DEBUG : fs cache: renaming cache item "d:\\test\\check\\dst" to be canonical "//?/d:/test/check/dst"
52d37544ca98287dd2b15f526ebfafa5  file.ext
2025/10/27 14:46:34 DEBUG : 2 go routines active

-------------
rclone check  d:\test\check\src d:\test\check\dst -vv 
2025/10/27 14:46:36 DEBUG : rclone: Version "v1.71.0" starting with parameters ["d:\\data\\rclone\\rclone.exe" "check" "d:\\test\\check\\src" "d:\\test\\check\\dst" "-vv"]
2025/10/27 14:46:36 DEBUG : Creating backend with remote "d:\\test\\check\\src"
2025/10/27 14:46:36 DEBUG : Using config file from "d:\\data\\rclone\\rclone.conf"
2025/10/27 14:46:37 DEBUG : fs cache: renaming cache item "d:\\test\\check\\src" to be canonical "//?/d:/test/check/src"
2025/10/27 14:46:37 DEBUG : Creating backend with remote "d:\\test\\check\\dst"
2025/10/27 14:46:37 DEBUG : fs cache: renaming cache item "d:\\test\\check\\dst" to be canonical "//?/d:/test/check/dst"
2025/10/27 14:46:37 INFO  : Using md5 for hash comparisons
2025/10/27 14:46:37 DEBUG : Local file system at //?/d:/test/check/dst: Waiting for checks to finish
2025/10/27 14:46:59 DEBUG : file.ext: md5 = 52d37544ca98287dd2b15f526ebfafa5 OK
2025/10/27 14:46:59 DEBUG : file.ext: OK
2025/10/27 14:46:59 NOTICE: Local file system at //?/d:/test/check/dst: 0 differences found
2025/10/27 14:46:59 NOTICE: Local file system at //?/d:/test/check/dst: 1 matching files
2025/10/27 14:46:59 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:                 1 / 1, 100%, Listed 1
Elapsed time:        22.6s
2025/10/27 14:46:59 DEBUG : 4 go routines active

-------------
rclone check  d:\test\check\src d:\test\check\dst -vv --download 
2025/10/27 14:47:02 DEBUG : rclone: Version "v1.71.0" starting with parameters ["d:\\data\\rclone\\rclone.exe" "check" "d:\\test\\check\\src" "d:\\test\\check\\dst" "-vv" "--download"]
2025/10/27 14:47:02 DEBUG : Creating backend with remote "d:\\test\\check\\src"
2025/10/27 14:47:02 DEBUG : Using config file from "d:\\data\\rclone\\rclone.conf"
2025/10/27 14:47:02 DEBUG : fs cache: renaming cache item "d:\\test\\check\\src" to be canonical "//?/d:/test/check/src"
2025/10/27 14:47:02 DEBUG : Creating backend with remote "d:\\test\\check\\dst"
2025/10/27 14:47:02 DEBUG : fs cache: renaming cache item "d:\\test\\check\\dst" to be canonical "//?/d:/test/check/dst"
2025/10/27 14:47:02 DEBUG : Local file system at //?/d:/test/check/dst: Waiting for checks to finish
2025/10/27 14:47:05 DEBUG : file.ext: OK
2025/10/27 14:47:05 NOTICE: Local file system at //?/d:/test/check/dst: 0 differences found
2025/10/27 14:47:05 NOTICE: Local file system at //?/d:/test/check/dst: 1 matching files
2025/10/27 14:47:05 INFO  : 
Transferred:   	   11.202 GiB / 11.202 GiB, 100%, 3.250 GiB/s, ETA 0s
Checks:                 1 / 1, 100%, Listed 2
Transferred:            1 / 1, 100%
Elapsed time:         3.4s
2025/10/27 14:47:05 DEBUG : 3 go routines active
4

Rclone Will check md5 sums without the --download flag. With it, it will compare the file contents byte by byte

5

thanks for that explanation, tho not sure what it means in practice?
check md5 versus compare the file contents byte by byte

in OP case and in my debug logs up above, the source and dest are both local.
there are no md5 sums to check, the only option is compare.

6

Thank you for all the replies so far.

Like @asdffdsa, I'm not so sure what to make of your answer.

I'm in the process of testing flags with --verbose --verbose. Without --download, I get something like this:

2025/10/28 09:16:40 DEBUG : Music/Bobby Conn.mp4: md5 = 800ed2df20f3f01cc1b6a8afb250fd1a OK
2025/10/28 09:16:40 DEBUG : Music/Bobby Conn.mp4: OK

Here's the exact command I'm running:

rclone check D:\ E:\ --exclude parent.lock --exclude '/$RECYCLE.BIN/**' --exclude '/System Volume Information/**' --checkers 1 --progress --verbose --verbose

(I mostly use rclone for scripting; that's why I use the long-form flags.)

7

Sorry if I wasn't clear.

When you use rclone check without the --download flag, rclone will run through the source and destination and check hashes of the files. So when you do rclone check on two local file systems, rclone will read the source file, compute the md5 read the destination file, compute the md5 and compare the two md5s. If they are equal then it declares the files equal and moves on to the next one.

When you use rclone check with the --download flag, rclone downloads matching files from the source and destination. It compares each byte of the source file to make sure it is identical to each byte of the destination file. (This is done in a streaming way in memory - it doesn't literally download the file). In the case of checking between two local destinations download the file means read it from disk.

So the practical upshot is that when doing rclone check from local to local it reads the entire file whether you use --download or not. If you don't use --download it compares the MD5s of the files and if you do then it compares the bytes. So theoretically --download is a stronger check but in practice you have about 0 chance of there being corruptions an MD5 doesn't detect.

I did a little test with 5G of files

$ time rclone check --download src dst
2025/10/28 10:21:14 NOTICE: Local file system at /tmp/dst: 0 differences found
2025/10/28 10:21:14 NOTICE: Local file system at /tmp/dst: 1999 matching files

real	0m3.894s
user	0m4.326s
sys	0m21.110s

vs

$ time rclone check src dst
2025/10/28 10:21:43 NOTICE: Local file system at /tmp/dst: 0 differences found
2025/10/28 10:21:43 NOTICE: Local file system at /tmp/dst: 1999 matching files

real	0m4.117s
user	0m22.691s
sys	0m13.814s

So they take about the same time. For local to local checks --download is slightly quicker (8%) and uses a lot less CPU and is more accurate, so I'd use that :slight_smile: But if you already did the check without --download then I wouldn't redo it.

8

Thank you for taking the time to provide such a detailed answer! :slight_smile:

1 Like
9

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.