when running the sync command, it shows me similarly named directories and an encrypted string as duplicated. i suppose this means that each one is duped, not that rclone is mistaking them for being dupes?
when I run rclone dedupe remote1_crypt:
i get the error: Failed to dedupe: Encrypted drive remote1_crypt: can't merge directories
Old thread, but I’m seeing this when I rclone copy to a WebDav remote. I did the first copy as a dry-run, and it listed the files which have changed. All looked good, so I removed --dry-run, and ran the copy. About 4GB and several hundred files were copied; all good.
I then re-ran the copy, just to sanity-check - I assumed I’d get no changes. However, instead, I got hundreds of files listed all with this error:
Duplicate object found in destination - ignoring
When I go onto the remote host and ls the folder, there are no dupes. If I do rclone ls on the remote folder, I see no dupes. So what is going on?!
The WebDav remote is a Synology NAS running WebDAV server, and I’m running rclone 1.43 for Mac, if that makes a difference.
I don’t know if it’s reproducible - it doesn’t seem to happen in the same way on my home network (where I am today). The errors I was getting yesterday were when I ran it over my VPN from a remote connection.
I’ll try it again tomorrow when I’m in the office. I sent you a log anyway, just in case it shows anything up.
However rclone does something called unicode normalisation on its strings so after unicode normalisation those strings are identical (which is what the user would expect since they visually look identical).
You might ask how these strings came into being? My guess is that these files have been on macOS at some point which stores denormalised strings in its file system by default and some tools (including older versions of rclone) don’t normalise the strings again.
I think the solution to this is to merge the contents of the directories.
Actually, I knew about that one, and it’s less of a surprise, since unicode always screws things up. What was confusing me was a bunch of new folders which claimed to have dupe files in - despite them clearly not (and no unicode in sight).
But sadly (or, I suppose, happily) I can’t repro it now, so it must have been a one-off. Thanks for looking though!