Rclone mount : Can i cache listing directory & filename without anything else?

Help and Support
1

Hi.

I want to use rclone mount and cache directory listing and files. I tried to use : --dir-cache-time 1000h --poll-interval 15s --cache-dir="cache" but it seems it does not use the cache-dir at all, it just cache on memory.

I need to use VFS cache option for that ?

Also, i don't want to wear my ssd, can i use a ramdisk to store vfs file caching data but keep directory listing / file metadata to a real ssd directory ?

2

can you post all command you tried?

3

I googled a little and just read that rclone does not support persistant directory caching, vfs is just for files.
Can you confirm ?

4

No. There is no split different cache data option. You can have all cache in memory or on disk. To use memory you have to create RAM disk and point cache there. But IMO this is waste of RAM unless you have very demanding workflow and plenty of memory

5

VFS-cache is persistent for given remote - you can reboot your computer and data still will be there

Here you find all details:

6

I tried something like
.\rclone.exe mount remote:/ localdir --dir-cache-time 1000h --poll-interval 15s --cache-dir="E:\cache-merge" --read-only --vfs-cache-mode full --vfs-cache-max-size 50G --buffer-size 100M --bind 192.168.1.124

And upon closing, it don't feel like directory listing were saved, i did a recursive list of my files with ls . -r but my vfs and vfsMeta folders are still empty.

So i'm not sure that directory listing are saved outside of memory, or maybe i'm missing an option ?

7

The caching of the directory and file structure is only memory.

There is no persistent cache for that. Once a mount is restarted, you have to list it all out again.

There is already a feature request out there for a persistent metadata cache.

That should not be confused with the vfs-cache-mode features for full/writes as that is for file caching to local disk only.

They are not connected to each other.

8

Ok, do you have a link to this feature so i can support it ?
It's a shame to have to do a full listing of your directory each time you relaunch rclone.
Also, i read this tip, i don't know if its still up to date :
Use : \rclone.exe mount s-merge:/ mountmerge --rc --attr-timeout 1000h --dir-cache-time 1000h --poll-interval 15s --cache-dir="J:" --read-only --vfs-cache-mode full --vfs-cache-max-size 8G --buffer-size 100M --bind 192.168.1.124 --rc-user=time --rc-pass=xxx --rc-web-gui
And then : .\rclone rc vfs/refresh --user time --pass xxx
To trigger a fast-list search and preload cache dir.

About VFS caching now, in VFS full mode, does the cache have to be bigger than files used ? Like if i set it to 10GB of ramdisk, what happen if i tried to open a heavy video file ?

9

It is called cache warming and works.

yes but not all remotes support fast-list - it will work regardless and warm up all cache

Yes - size of cache can exceed what you set in vfs-cache-max-size, so it can get as big as your biggest file. As per docs:

If using --vfs-cache-max-size note that the cache may exceed this size for two reasons. Firstly because it is only checked every --vfs-cache-poll-interval. Secondly because open files cannot be evicted from the cache.

10

And i guess there is no option to force --vfs-cache-mode off for files bigger than an user defined size ?

11

Your guess is absolutely correct.

12

Ok, so i'm gonna buy a cheap MX500 to use a consumable.

13

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.