Rclone poor man's deduplication?

I don't have any storage system with builtin deduplication, but I mirror several sites with a high level of duplicate files. Is there any way of using rclone in a mirroring workflow that includes deduplication?

For example, there are several utilities that replace duplicate files with symlinks. If rclone were to encounter a symlink pointing to file A would it interpret it as a duplicate of the remote file A?


best to post questions using the help and support template and answer the questions

Hi, this isn't a specific help request on how to use rclone, that's why its in the Features section, which has no templates I believe.

I'd only like to know if rclone recognizes hardlinks, symbolic links, or Windows shortcuts as placeholders for the original files.

There are two flags dealing with links - rclone will ignore them by default. These work on the local filesystem only.

-L, --copy-links Follow symlinks and copy the pointed to item.
-l, --links Translate symlinks to/from regular files with a '.rclonelink' extension