Moving files from one Google Workspace Shared Drive to another. Possible bug?

Although if you do a few cycles, you'd lose a file.

If I move again, the dupe isn't there and my dupe overwrites my destination so the original file is gone I suppose.

My use case is always on a OS so I never hit dupes even though they are allowed. To get a dupe, you'd have to do it with some purpose via the WebUI generally.

Ideally, rclone would detect if it is a duplicate name or name+content and word the error accordingly.

I don't think there is any way to do that right now, as you can see in my examples for some files rclone cannot produce an md5sum, nor the file has a size (it is -1). Therefore no way to know for rclone, as it is now.

So something like:

ERROR: Stargardt_Summary .xlsx: Duplicate folder/file name. Content might differ. Use rclone dedupe to find and fix duplicate names. 

Might not be the perfect wording, but for sure catches attention and forces the rclone user to go a bit deeper and understand what is really going on.

My 2 cents.

Thanks, I like your addition, makes it clearer. I will see if it can be easily changed one of the coming days.

1 Like

I can try to give a suggestion also for the wording of:
https ://rclone.org/drive/#duplicated-files

I mean, it took me a full day to figure all this out, why not bring it back to the community.

Would this be the right place?

Yeah, any wording or things like this, we can generally make those changes as I've seen the duplicate file thing for a few years now so it's all very up front to me so the wording makes sense.

If you have any updates or whatnot, you can either share them here, or even there are a number of ways to submit the change and update it.

That's the best part of open source as everyone can help on it too. I'm not a developer at all so I try to help/ask/answer as I've done IT for too many years..

That would be great!

I suggest you create a new topic of type Feature with a reference to this thread, that will give it the right attention.

One way to do this with minimal overhead is to simply append a GUID to each duplicated file. So if Rclone finds three copies of sample.txt, it would copy them like this:

  • sample.txt
  • sample_$GUID1.txt
  • sample_$GUID2.txt

That behavior is probably not for everyone, so it would be desirable to be on a flag, but it would eliminate duplicates as a problem in scenarios like mine, where running dedup is not feasible.

1 Like

I like that idea.

I don't think the naming is the primary challenge in handling duplicates.

It is because rclone like 99.99% of all programs is built on the core assumption that path and filename uniquely identifies a file. Think about a command like this:

fileHandle = openFile("hello.txt")

Most programmers would expect it to open a single file and return the corresponding fileHandle. It would require quite some reengineering in the core of rclone to make it treat fileHandle an array of fileHandles. The next questions after doing that is how to compare (when some backends don't support hashes etc.), how to list, etc. etc.

This GitHub issue illustrates how apparently simple and quick fixes like renaming in the backend suddenly opens for a lot of other questions and difficulties:

I am not saying it is impossible, just that it would require substantial sponsorship or development contribution.

It may be less demanding to make a dedicated Google to Google copy/migration tool from the ground.

1 Like

I have made a feature proposal here:
https://forum.rclone.org/t/improved-messages-on-duplicate-files-during-copy-sync-move/35696

Saw it. That is great! Let's see what the others think about it. It is going to be interesting :slight_smile: