I have tried both v1.47.0 and v1.47.0-122-g64fb4eff-beta (The beta was needed for dedupe on crypted storage).
I'm currently doing a very large sync from Google Drive to my local storage (30TB, 500K files of different sizes).
I am however hitting this "Duplicate object found in source - ignoring" on multiple files.
I have been running dedupe on both the encrypted on decrypted remote, but it is not finding the files.
One duplicate would be GoogleDriveCrypted:a/b/c/d/e/f/file1.ext
I ran the sync as rclone sync GooglDriveCrypted:a/b ./
I ran dedupe as rclone dedupe GoogleDriveCrypted:a/b
I'm not sure if it's a bug, but I would think that, as it is complaining about duplicates but not finding them.
Also when running rclone ls GoogleDriveCrypted:a/b/c/d/e/f/file1.ext it only returns a single file1.ext not sure if that is to be expected even with duplicates?
Are you on Windows or Mac with a case insensitive file system?
You might have two files which are "HELLO.txt" and "hello.txt" which google will store fine, but will register as duplicates if transferring to a case insensitive file system.
If that isn't the problem then use rclone lsf to list the problem source and destination directories and compare.
Posting a log (or parts of) with -vv would be helpful too.
Yes a single file will always be returned here. If you have duplicates then which one is returned is undefined.
You are right that it is a case problem, however I am syncing onto a ZFS pool which is case sensitive (OpenZFS on OS X 1.9.0), so that should be allowed? It seems like it is detecting them as duplicates even though they are in fact not duplicates and should be fine on that FS.
I also tested, I'm able to create 2 files which only differs by case just fine on the pool.
Could it be that rclone cannot properly detect the case sensitivity on that configuration of filesystem?
rclone assumes that everything on OSX is case insensitive. It isn't easy to detect whether a file system is case sensitive or not without writing a file to it, which I considered but decided that it would be unexpected for the user.
I made a beta with a --local-case-sensitive flag in this beta which overrides that default - can you give it a go?
