Rclone dedup between 2 different paths

Help and Support
1

Hello, I'm wondering if it possible to use rclone dedup to compare 2 paths.

My use case is the following, I'm using rclone to sync 2 directories (let's call them source and target), and sending removed files to 3rd (backup) one using --backup-dir

The problem is, sometimes files in source where not removed, simply moved to a diferent path. So I would like to do a hash comparision between source and backup, and if it already exists in source, then remove it.

Is this possible?

rclone v1.60.1-DEV
- os/version: ubuntu 24.04 (64 bit)
- os/kernel: 6.11.0-24-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.22.2
- go/linking: dynamic
- go/tags: none
2

welcome to the forum,

that is a many years old, custom compiled dev version from an out of date repository.

  1. uninstall that old version
  2. https://rclone.org/install/#script-installation
  3. run command again
  4. if still need assistance, start a new topic, and answer all the questions
3

Version updated

rclone v1.69.1
- os/version: ubuntu 24.04 (64 bit)
- os/kernel: 6.11.0-24-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.24.0
- go/linking: static
- go/tags: none

the "questions" are irrelevant, I'm not having an issue with rclone, I just want to know if it supports what I need or I need to look for a different tool.

4
  1. rclone check src: dst: can output a list of files.
  2. feed that list to rclone delete src: --files-from

or maybe

  1. rclone dedupe src: dst: --dedupe-mode list
  2. feed that list to rclone delete src: --files-from

or maybe

  1. rclone mount src: /path/to/mount/src/
  2. rclone mount dst: /path/to/mount/dst/
  3. point any dedupe tool to /path/to/mount/src/ /path/to/mount/dst/

or maybe
https://forum.rclone.org/t/big-syncs-with-millions-of-files/40182

5

Rclone dedupe feature is for special cases when one remote allows for duplicates to exist.

I would use tool called fclones (FOSS, you can Google it easily) and rclone only to mount remote cloud storage if needed.

Have a look also at this thread as an example of using fclones:

6

thank you, I'll look at those suggestions