VFS cache: pin / warm / cold controls per file (prototype patch)

saipr0 · October 5, 2026, 5:41am

My home server has ~500 GB; Google Drive is cheap bulk storage but not private. So I keep everything in Drive behind rclone crypt and serve it through Nextcloud, using the server's disk as a local cache. I want to choose which files stay local (hot) and which live only in the cloud (cold).

With --vfs-cache-mode full, eviction is age/size LRU only. For a Nextcloud-over-rclone setup I wanted per-file control:

pin: keep a file or folder local permanently
warm: download a whole file now
cold: drop a file's cached data now

I prototyped this as a patch on v1.75.1:

github.com/saipr0/workspace-storage

rclone/workspace.patch

main

diff --git a/vfs/dir.go b/vfs/dir.go
index 0d24195..6ea98ad 100644
--- a/vfs/dir.go
+++ b/vfs/dir.go
@@ -1111,6 +1111,12 @@ func (d *Dir) Remove() error {
 		fs.Errorf(d, "Dir.Remove failed to remove directory: %v", err)
 		return err
 	}
+	if d.vfs.cache != nil {
+		if err := d.vfs.cache.WorkspaceDelete(d.path); err != nil {
+			return err
+		}
+	}
+
 	// Remove the item from the parent directory listing
 	if d.parent != nil {
 		d.parent.delObject(d.Name())
@@ -1178,6 +1184,12 @@ func (d *Dir) Rename(oldName, newName string, destDir *Dir) error {
 		fs.Errorf(oldPath, "Dir.Rename error: %v", err)
 		return err

This file has been truncated. show original

Pins/jobs persisted in a JSON policy file next to vfsMeta
Background worker that warms/evicts files one at a time
New rc command workspace/control
Cache cleaner skips pinned and in-progress files
Upstream changes are small (5 files, +41 lines); the rest is new files with tests

Nextcloud view -

Nextcloud Files on top of the patched rclone: = not fully cached, = fully local, = pinned

Per-file status overlaps with rclone issue 8779 (vfs/dir-status); I'd align with that rather than duplicate it. The part I haven't seen proposed is the control side. Earlier asks on this forum: "Pinned persistent cache?" (t/25520) and "rc cache/fetch for vfs?" (t/45468).

The idea and design are mine; I used Codex to quickly prototype it and see if it was feasible. I'm now reviewing the code line by line. I'm a young dev doing this partly to learn rclone's internals, so any pointers on the code are welcome too.

Questions:

Is this approach sane, or is there a better way to get pin/warm/cold without modifying rclone (existing flags, rc commands, or a wrapper around the stock cache)?
If it does need rclone changes, would pin/warm/cold make sense as rc commands next to vfs/dir-status?

saipr0 · October 6, 2026, 6:50pm

Update: after reviewing the code I ended up changing quite a lot, so the patch at the same link is now mostly rewritten.

How it works now:

Pins are saved in a small JSON file next to vfsMeta. Pinning a folder covers everything inside it, and the most specific entry wins, so I can pin a folder and then unpin individual files inside it. Pins follow renames and get removed when the file is deleted.

Warming just reads the file through the VFS so the full cache keeps it. The cleaner skips pinned files in removeNotInUse and purgeClean.

Cold is now an action instead of a mode. It removes eligible cached copies immediately. If a file is open, still uploading, or in the handle-caching grace period, the normal cleaner removes it once it's free.

New pin and retry requests are checked against --vfs-cache-max-size. The size check and accounting happen together under the cache lock, so concurrent requests can't reserve the same free space. Failed downloads keep their space accounted for until unpinned.

New rc commands:

vfs/pin: accepts path or paths, with mode pinned or unpinned
vfs/evict: unpins and releases cached copies
vfs/retry: retries warming a pinned path
vfs/status: lists a directory's entries with pin/cache status, failure and pending-eviction flags, mixed folder policies, and how much pin space is left

The changes to existing rclone files are small: about 65 added lines across dir.go, file.go, vfs.go and vfscache/cache.go, plus the new rc commands in vfs/rc.go. Everything else is in new files, around 560 lines of code and 440 lines of tests.

Questions:

Would something like this be welcome upstream? Do separate rc commands make sense for pin/evict/retry? For status reporting, should I align with the proposal in issue 8779?
To warm a file I just read it fully through the VFS so the cache keeps it. Is there a better way to pull a whole file into the cache?