I'd like to use the VFS cache but limit how much space a single folder can use...
is the best way to do this still usin a rclone union and then set the local cache for each union as I want to be?
I usually prefer to have less layers for performance reasons, as I expect rclone to use ~40 Gbps out of this mount.
It's a raid 0 of nvme ssd drives
Is that a specific folder, or any folder you'd like to limit?
The VFS cache sits on top of the union so I don't think this idea will work.
Maybe a program which scans the files in the cache via the API and removes the ones you don't want would work?
I'd like to specify a single folder that can't go over X TBs, the other folders can be unlimited
The union wouldn't have a vfs cache only the remotes that it's uniting, wouldn't that work?
Also I have been thinking and there are a lot of good reasons to be able to set specific amount of disk space per folder, or in a single folder. And maybe even specific cache times for different folders or a specific folder with another global settings
Big folders with lots of small files - It would kill performance in HDDs and even in NVMEs and those files would be easy to download on the fly in high speed networks
Rclone cache takes a long time to start if they have a lot of items.
Some folder may have files that are accessed only a few times, while others are accessed a lot more. And those files will take disk space without need.
And probably a lot of other use cases I can't think right now
Lots of things rclone could do with the cache - that is for certain!
Some different cache policies would help - like in this issue: VFS Cache: control caching of files by size · Issue #4110 · rclone/rclone · GitHub
I guess if rclone had a slightly richer API then you could implement some cache cleaning policies externally.
Or maybe have a cache policy plugin.
I like that idea, it seems similar to what I said indeed.
I do think rclone cache needs to expand a lot because it's really useful but to be used in a massive scale like I do it needs to be like a cloudflare cache, you can tweak a lot of stuff.
How we can make that happen ?
Sorry for the delay replying.
I guess if you'd like some specialized features for the cache I could do those in a contract? Would that be interesting?
Can you message me with a form of contact so we can discuss this further?
I'm moving all my servers to 10 gbps and your help would sure help a lot !
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.