Flag to merge duplicate folders

Ran a search and didn't see this, although I'm guessing I missed it as I suspect this has been asked for before. My apologies if this is a dupe request.

I don't know what the feasibility or complexity of this is, but it would be nice to include a flag that uses a combination of the functionality of the union remote plus the dedupe option for rclone mounts.

e.g. If I mount remote A with:
/root
/FolderA
File1.txt
File2.txt (older)
/FolderA
File2.txt (newer)
File3.txt
/FolderB
etc.
and I mount said remote at the root level of the remote with a specific flag, e.g. --merge-directories and --duplicate-file-mode=newest
an directory listing on my mount of FolderA would show File1.txt, File2.txt and File3.txt, with file2 being the newer version. This flag could work with other similar dedupe flags like largest, oldest, etc. Alternatively, if this is easier, you could make union remotes apply the same rules within a single remote member. So if I make a union remote and add RemoteA: as a member or even the only member, it respects the merger rules for duplicate folders/objects within said remote.

I recognize that deduping the remote would resolve these situations and it has fixed the issue I've come across when manually run. The issue is, at least in my situation, I've run into a race condition in which if a folder is moved to from local to rclone remote with a small file in it (meaning the move finishes in seconds), followed quickly by a second move with the same folder name, the first move doesn't always register fast enough, and a duplicate folder is created for the second move. Since the mount sees the first folder created, the second item moved never appears until a dedupe is performed.

Just a thought. Thanks for your consideration.
-Ed

You can check out the issues:

and maybe

Duplicates get a bit odd as not every provider can have them. Google allows, but lots others don't. I'd guess in your example, you have Google as well?

ahh I see. Yes in my use case it's using a Google remote. Thanks for the reply!

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.