Path-agnostic dedupe?

Help and Support
1

What is the problem you are having with rclone?

I want to search for identical files (preferably by hash, not name) which might be in different path locations.
'''
dedupe --dry-run
'''
Only marks a file if it's in the same path as its duplicate. I want to mark all sets of duplicate files, even if they are in different file locations.

What is your rclone version (output from rclone version)

1.53.3

Which OS you are using and how many bits (eg Windows 7, 64 bit)

on a google F1 instance running Ubuntu 18, I'm pretty sure it's 64 bit

Which cloud storage system are you using? (eg Google Drive)

Google drive and Mega

2

That's not a duplicate in rclone unfortunately.

A duplicate is the same file/directory and it doesn't search paths. The reason dedupe exists is mainly for Google Drive.

You'd probably want to find a utility to do something like that rather than rclone.

3

In the latest beta we recently merged an extension to rclone dedupe

You can add the --by-hash flag which will dedupe by hash of the file.

Download the beta and run rclone dedupe --help for more info.

1 Like
4

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.