Cache Backend being removed

[tcf909 (T.C. Ferguson)]

I read in one of the Github issues that "Cache" is going away at some point. I was hoping you could elaborate on what that means and why so we can plan appropriately.

Thanks

[calisro (Rob)]

Does this help?

https://rclone.org/cache/#status

May I ask why you use the cache today?

[ncw (Nick Craig-Wood)]

I want to have a more integrated replacement. I expect we could do backwards compatibility also so the cache config was applied to the new subsystem.

[tcf909 (T.C. Ferguson)]

Makes sense.

How are you handling write reliability (offline uploads), persistent meta caching, and some of the other things Cache does that Vfs doesn't (that I'm aware of)?

[tcf909 (T.C. Ferguson)]

or am I misunderstanding that VFS will replace Cache?

(maybe your rebuilding the Cache backend and bringing it in house?)

[tcf909 (T.C. Ferguson)]

Calisro:

We use Cache for many things and hundreds of mounts. We spent a great deal of time experimenting and VFS never cut it (especially with Crypt in the mix).

Some of the things we rely on (albeit wish were better) are offline uploads, separate index and chunk destinations, persistent meta storage, hot block caching (sub file reads).

[Animosity022]

Take a look at:

Testing for new vfs cache mode features

I'm in the process of completing the changes to the vfs cache This will have these 3 major new features (along with a host of bug fixes!) --vfs-cache-mode full which will only download the parts of the file you need I envision this will become the preferred way of using the vfs cache --vfs-cache-mode writes will do delayed writeback of files this will help with explorer lockups on windows this will help with downloads and renames in quick sucession (eg downloading from a web browser) r…

For a rough summary, it's being integrated into the base VFS mount so things in cache are being moved into the 'standard'. If you have very specific use cases, feedback would be welcome along with testing!

[tcf909 (T.C. Ferguson)]

Just went through the post and quickly browsed through the vfs branch. The branch looks really promising. The new VFS functionality doesn't appear to be missing any features that Cache brings to the table. Did I miss something?

[Animosity022]

I don't think so.

The only thing missing per se, Plex integration, which had a great idea, but not really needed.

Other than that, I think the future is looking bright!

[tcf909 (T.C. Ferguson)]

We never enabled Plex integration as it caused other issues where a mount might be used for other things beyond media.

[nle]

@tcf909 Care to elaborate a bit on your use of rclone? Sounds like a interesting way to use it.

[ncw (Nick Craig-Wood)]

@tcf909

Some of the things we rely on (albeit wish were better) are offline uploads, separate index and chunk destinations, persistent meta storage, hot block caching (sub file reads).

The VFS cache will help with some of that - offline uploads should work. The metadata is kept as small files on the disk so I don't think will be a problem unlike the cache db. It will do sub file reads too.

One thing it doesn't do yet is cache metadata persistently (other than for files it has cached). I'm planning on doing this eventually but one thing at a time!

[Lex]

Running beta 044 and it is smooth as butter.

Any bugs stopping it from merging with mainline beta?

[Animosity022]

See:

Testing for new vfs cache mode features

That is great news! I've been testing it on my home internet which isn't the quickest and it is as fast as everything else. I should do some local testing too I think.

[Lex]

Hi Anim. Yes - I meant to post in that thread... bounced tabs by accident.