How to dedupe a union

Help and Support
1

As i was saying, I tried to dedupe a union and was unsuccessful.
:orange_circle: Union remote consist of 6 mega cloud root directory
:orange_circle: rclone version : rclone v1.58.1

  • os/version: ubuntu 20.04 (64 bit)
  • os/kernel: 4.4.0-1101-aws (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.17.9
  • go/linking: static
  • go/tags: none

:orange_circle: command : dedupe --by-hash unionx:
Result : 2022/05/06 19:56:57 Failed to dedupe: union root '' has no hashes

:orange_circle: command : dedupe unionx:
Result :2022/05/06 20:20:54 NOTICE: union root '': Can't have duplicate names here. Perhaps you wanted --by-hash ? Continuing anyway.

:orange_circle:My Goal : I want 6 mega account store unique files without any duplicate.

Please suggest me solution

1 Like
2

Dedupe was primarily created for Google Drive and remotes that allow duplicates as mounts/most OS'es don't allow that to be a thing.

On a mount or union (in this case), you won't get presented duplicates as no OS generally works with that.

You could generate a file listing from each and compare them I'd imagine.

3

Thank you @Animosity022
Then is there any way I can remove duplicates from 6 mega remote without listing and deleting them one by one?

4

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.