Check if a file in a specified folder exists anywhere on remote

[disconnect5852]

Hi!
I have an SD card with many mp4 videos, and i want to check which files are already backed up to an rclone remote. For this rclone check would work, but these files can be in many subfolders in remote, so i want to check for the files in all subfolders too on remote, by size or hash, as the names, and the file dates can be different.

How to do it?

[kapitainsky]

Try sync with --track-renames and --dry-run. I think it should reveal already existing files in the debug logfile.

Alternatively mount your remote and run de-duplicator of your choice against your SD card.

I would use fclones as it allows to limit amount of data read from remote, see this old thread:

Rclone dedupe on crypt remotes?

I would mount encrypted remote with --vfs-cache-mode full --vfs-read-chunk-size 1M and use something like fclones. then you could run: fclones group . --max-prefix-size 1MB --max-suffix-size 1MB --skip-content-hash It would only read and hash 2MB (1MB from the beginning and end of a file) from every suspected duplicate (where length is the same) and thanks to --vfs-read-chunk-size 1M mount option only 2MB are read from remote per file.