Rclone check - SUM

Looking at the new Rclone check SUM support - trying to get away from my own script that generates checksums... seems like I can use rclone check SUM to do this for me?

--checkfile HASH

I'm not sure what you are asking.

The check help page is here:


Is there a link somewhere to what the exact syntax is of the SUM format ?

If you supply the --checkfile HASH flag with a valid hash name, the source:path must point to a text file in the SUM format.

All written up here:

you can create a sum file using rclone

for example,
rclone hashsum md5 . --output-file=sum.txt -vv

rclone check sum.txt . --checkfile=md5 --one-way -vv

I'm working on verifying this but does anyone know if the filePath in the SUM file can be absolute? or just it just need to be relative? like it is in the output of

rclone hashsum md5 . --output-file=sum.txt -vv

ie the command above outputs:
e4cb2eafeaade82c723af3a058cec2b2 Pictures/2021-02-07 17-45-40.jpeg

where as I have in my SUM file:
E4CB2EAFEAADE82C723AF3A058CEC2B2 C:\\Users\\Tony\\Pictures\\2021-02-07\ 17-45-40.jpeg

rclone does not support absolute paths in SUM files.
SUM paths are always treated as relative to the remote argument of the hashsum, checksum, or check command.
absolute paths will make rclone print a error.

please keep in mind that rclone is a cloud-first tool and does not have a notion of absolute paths. all paths are relative to some "remote".
even local paths iterated during operations are treated as relative to the local path supplied on command line which constitutes a per-command "local remote".

did some quick testing and this worked for me.

dir of the folder containing the files.

b:\test>dir /b

d41d8cd98f00b204e9800998ecf8427e test/file.txt

rclone command
rclone check sum.txt b:\ --checkfile=md5 --one-way -vv

so i would test the following

  1. change
    C:\\Users\\Tony\\Pictures\\2021-02-07\ 17-45-40.jpeg

  2. use a command such as
    rclone check sum.txt c:\ --checkfile=md5 --one-way -vv


thanks - curious though if I'm running on windows why do I need to change my path slashes? I guess its part of the cloud first ?

in my testing, need to use /, not \
tho it seems that rclone silently ignores a path that uses \, no warning, no error...
the error.txt created is empty, a zero byte file


d41d8cd98f00b204e9800998ecf8427e  test\file01.txt
d41d8cd98f00b204e9800998ecf8427e  test/file02.txt
rclone.exe ls b:\test 
        0 file01.txt
        0 file02.txt

rclone.exe check sum.txt b:\ --checkfile=md5 --one-way -vv --include="/test/**" --error=error.txt 
DEBUG : rclone: Version "v1.56.0" starting with parameters ["c:\\data\\rclone\\scripts\\rclone.exe" "check" "sum.txt" "b:\\" "--checkfile=md5" "--one-way" "-vv" "--error=error.txt"]
DEBUG : test/file02.txt: md5 = d41d8cd98f00b204e9800998ecf8427e OK
NOTICE: Local file system at //?/b:/: 0 differences found
NOTICE: Local file system at //?/b:/: 1 matching files

In many clouds, say S3, the file names can contain *?:\ and so on without limits.
The only greatest common divisor common path separator is /, so rclone follows the majority.

Probably in future we can add special options for check and checksum like --skip-prefix-dirs N that will bite off the given number of leading path directories in the SUM file like patch -p (and probably something to convert windows slashes). Fancy submitting a patch or at least a feature request?

I oracle that if you omit --one-way then rclone will cry aloud that it can't find the test\file01.txt file with backslash in the name (if my memory doesn't fail me, --one-way means ignore files missing on one side of the check).

ok another question here ... I want to have rclone check my local sum file against my rclone remote - is that possible?

yes, that is possible

rclone hashsum md5 .\source --output-file=sum.txt

560f9e8172f5d359e6cb2205adbcc5df file.txt

rclone checksum md5 sum.txt remote:testfolder01 -vv 
DEBUG : rclone: Version "v1.56.0" starting with parameters ["c:\\data\\rclone\\scripts\\rclone.exe" "checksum" "md5" "sum.txt" "remote:testfolder01" "-vv"]
DEBUG : Creating backend with remote "sum.txt"
DEBUG : file.txt: md5 = 560f9e8172f5d359e6cb2205adbcc5df OK
NOTICE: S3 bucket testfolder01: 0 differences found
NOTICE: S3 bucket testfolder01: 1 matching files
1 Like

well, i was taught in greek mythology high school class.
never trust an oracle that gives a correct answer the first time...

Alright so I had rclone generate a sample sum file:

94918a22d08f0af48da351e517ad9ae6  NTUSER.DAT{53b39e88-18c4-11ea-a811-000d3aa4692b}.TM.blf
                           ERROR  NTUSER.DAT
6f7d4c882b008493c20d4b590e548de3  _netrc
                           ERROR  ntuser.dat.LOG1
                           ERROR  ntuser.dat.LOG2
6fc234ad3752e1267b34fb12bcd6718b  ntuser.ini
b4f52fc75ea9380f652e0bb17055e1f4  NTUSER.DAT{53b39e88-18c4-11ea-a811-000d3aa4692b}.TMContainer00000000000000000001.regtrans-ms
c5373647232233e265896e88703b1ce2  .gitconfig
59071590099d21dd439896592338bf95  NTUSER.DAT{53b39e88-18c4-11ea-a811-000d3aa4692b}.TMContainer00000000000000000002.regtrans-ms
293e53c7631fce3a3ab033458aa85a2c  sum2.txt
c0aec59d3c84d2d7928e3bffc9999f7b  sum.txt
793bc044a231708c798287dc6a01924b  summmm.txt
9e36cc3537ee9ee1e3b10fa4e761045b  Desktop/desktop.ini

If I now use that file to try and run rclone check it complains that there's invalid lines??

rclone check C:\Users\Tony\sum2.txt remote:/ --checkfile=md5 --filter-from "C:\Program Files (x86)\rclone\filter-file.txt"
2021/08/24 17:06:15 NOTICE: sum2.txt: improperly formatted checksum line 1
2021/08/24 17:06:15 NOTICE: sum2.txt: improperly formatted checksum line 3
2021/08/24 17:06:15 NOTICE: sum2.txt: improperly formatted checksum line 4
2021/08/24 17:06:16 ERROR : My Videos/.DS_Store: sum not found
2021/08/24 17:06:16 ERROR : My Videos/100_1324.MOV: sum not found
2021/08/24 17:06:16 ERROR : My Videos/100_1325.MOV: sum not found

post the command used to generate the sum file.
post filter-file.txt

rclone hashsum md5 . --output-file=sum2.txt -vv


- /My Pictures/Scans/**
- /My Pictures/Mom/**
- /My Videos/google_unsupported/**
+ /My Pictures/**
+ /My Videos/**
- /**
- **.xmp
- **.DS_Store
- **.ini

what is the purpose for creating the hashsum, what you do plan to with it?

I want to check the files on my remote against the SUM file. Any that exist on the remote but not the SUM file I'll copy locally for further processing.