Speeding up checks of big files

[kapitainsky]

mount your remote and use whatever program you prefer, for example fclones and partial hashes:

Rclone dedupe on crypt remotes?

I would mount encrypted remote with --vfs-cache-mode full --vfs-read-chunk-size 1M and use something like fclones. then you could run: fclones group . --max-prefix-size 1MB --max-suffix-size 1MB --skip-content-hash It would only read and hash 2MB (1MB from the beginning and end of a file) from every suspected duplicate (where length is the same) and thanks to --vfs-read-chunk-size 1M mount option only 2MB are read from remote per file.

It is about deduplication but you could creatively use it for check too - as long as every file has duplicates it means it was transferred... Simply include both mount path and source path in fclones params.