i have an Idea of a new Mode let's call it "--vfs-cache-mode light"
What should this Mode do? It should pre-download small parts / chunks of the files on a mount.
Why? Other Modes wörks great for bigger files, but not for small files like Music files wich Need gapeless Playback between two files, there are loading to slow from Cloud (gdrive).
So my idea, why not preloading the first parts of the files. If we browse on a dir on a mount simple preload some small parts of the file instead of downloading the whole file in mode full.
It works with a Cache remote, but whould be great if it will works on vfs-cache too.
The whole VFS backend is being rewritten to incorporate more of the cache backend.
You'd think with a global lockdown @ncw would be done already with it but he still needs more time
Just keep checking back as I know he's getting closer to releasing it in beta form and it somewhat merges a lot of the cache functionality into stock VFS.
You've described pretty much what the reworked vfs code will do! It will download parts of files and cache them on disk for you. You'll need --vfs-cache-mode full for this.
So let's say you have plex/whatever doing media scans. You have a 5 GB video file. Plex reads the start and the end to work out what kind of file it is and generate a thumbnail (say - I don't really know what plex does!). The new cache mode will just download the parts of the file that plex reads and store those on disk in a sparse file. This file will look like it is 5 GB in size but it will only actually have the data plex has read in it and so will only take a few MB of disk space. There will be another small metadata file recording what data rclone has read so it knows it doesn't need to read that again unless the upstream object changes.
I think when all the wrinkles are worked out, --vfs-cache-mode full should be the best cache mode to use. It won't download the whole file before starting (like it does now) and it will deal really well with seeking, random writes etc.
Anyway I have it basically working but I'm concentrating on getting 1.52 out the door before I put it out for beta.
It'll be announced properly when he's ready to test so if you think about it, the more he has to respond to when is it ready is time he could be coding it