UnionFS Improvements

Dimtar · October 26, 2019, 2:44pm

Hi all.

I hope this doesn’t come across as ungrateful, rclone is easily one of the best tools around and I am immensely thankful for it.

In saying that it has been roughly a year since this discussion and I wondered if any progress has been made on it.

github.com/rclone/rclone

rclone union mount allow more than one writeable remote

opened 08:16AM - 12 Nov 18 UTC

closed 11:39AM - 25 Jun 20 UTC

iamjrock

enhancement Remote: Union

#### Output of `rclone version` v1.44, however this is a request for a new feat…ure. #### Describe the issue This feature request has been discussed at length [here](https://forum.rclone.org/t/advantage-of-new-union-remote/7049). rclone _union_ mounts currently have the restriction that only the last remote can be written to (all other remotes are read-only). If we can remove this restriction then many rclone users who currently deploy unionfs or mergerfs could instead consolidate down to just using an rclone union mount. This is especially helpful for Windows/OSX users as there are no readily available union filesystem solutions like unionfs/mergerfs for these operating systems. Whilst removing the read-only restriction would accommodate many scenarios, the very common use case that we would like to achieve is: - location-0 is a local temporary folder and has the highest read priority. location-1 is a cloud remote with most of our storage. - New files: When our local apps create new files, they are written to location-0 without any latency. - Moved/Renamed files: -- If the original file exists only on location-0: just mv within location-0 -- If the original file exists only on location-1: just move (server-side) the file in location-1 -- If the original filename exists in both locations: mv the file in both locations (server-side mv for location-1). - Updated files: If the file is opened and saved, or replaced with a file of a larger/smaller size (same filename): -- If the original file exists only in location-0: just overwrite the original file in location-0 -- If the original file exists only in location-1: leave the old file in location-1 and write the new file to location-0 -- If the original file exists in both locations: leave the old file in location-1 and overwrite the old file with the new file in location-0. - Deleted files: Delete from both locations. This would then be cleaned up periodically with the following commands: - rclone move any file from location-0 to location-1 that is not identical (filename, subfolder, size) in both locations; then - delete any remaining files left over in location-0 (these are probably files that have the same filename and size in both remotes). This periodic cleanup could be managed separately from this feature request, or included as union mount functionality. The rclone forum thread also discusses various rclone.conf schemes that might be used to define the behaviour.

I see now there has been more discussion on Github. Mods can close this if need be otherwise maybe some discussion here will be productive.

thestigma · October 26, 2019, 3:20pm

The issue is not dead, don't worry.
I've been in talks with @ncw about this (around the time I updated the topic), and last I heard from Nick the plan is to take a serious look at it after 1.50 releases (which should be happening quite soon I think).

So with just a little more patience (and a lot of appreciation Nick's way for being such an awesome guy) I think this will work out

But as always, no hard promises or deadlines...

thestigma · October 26, 2019, 3:30pm

Aaaand there we go hehe, 1.50 came out 4hrs ago it seems:

system · January 24, 2020, 3:31pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.