Why aren't .cache/rclone files being removed?

What is the problem you are having with rclone?

Sorry, I'm having difficulty understanding VFS caching. I thought that files in .cache/rclone would be automatically removed, depending on vfs-cache-max-age and other settings.

To be specific, I've mounted using vfs-cache-max-age 2h (for example) and the files in .cache/rclone are still there well after 2 hours. I'm not sure if this is due simply to my misunderstanding of how VFS caching works, or if there is some other issue (whether due to my usage of the commands, or something else).

I did at some point manually rm -rf the .cache/rclone directory which I've subsequently read is ill advised (if this is the source of the problems, then is there a way to reset/fix things?).

I suppose a related question is, how can I force Rclone to remove files from .cache/rclone?

Thanks much in advance!

Run the command 'rclone version' and share the full output of the command.

rclone v1.57.0-DEV
- os/version: fedora 34 (64 bit)
- os/kernel: 5.14.14-200.fc34.x86_64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.16.11
- go/linking: dynamic
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Backblaze B2

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount --vfs-cache-mode full --vfs-cache-max-size 100G --vfs-cache-max-age 2h server_name:/path_to_folder/ Downloads/b2/

hello,
--- best to run the official rclone client, unless there is specific reason?
--- --vfs-cache-max-age is a soft limit, if the files are open/in-use, then rclone will not remove them.
--- without a debug log, no way to know for sure.

might re-start the mount and look at the debug log for problems.
or
delete the entire vfs file cahe and restart the mount

Thank you for your replies! Sorry, what do you mean by official rclone client? I'm using the rclone command-line tool (which I installed via the Fedora repository).

you are using some custom development version, not compiled by the rclone team.

this is the official client which can be found only here

rclone v1.57.0
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.10.60.1-microsoft-standard-WSL2 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.17.2
- go/linking: static
- go/tags: none

notice the differences
--- v1.57.0 versus v1.57.0-DEV.
--- go1.17.2 versus go1.16.11
--- static versus dynamic

If you stop the mount and rm -rf the cache area, there aren't any problems.

If you have a running mount and rm the cache area, it'll complain a bit as files it expects to be there are not.

The cache will sit as the max size pretty much until it goes over the max size. Once it hits the max size, it removes the oldest / files not in use from the cache.

It's all sparse files so once you set the max size you want, you just set and forget it as there's really no reason to muck with it.

1 Like

the rclone authors are clear about this, do not use repositories.
you are using some custom development version, not compiled by the rclone team.

I wasn't aware of this! Thank you for bringing this to my attention.

The cache will sit as the max size pretty much until it goes over the max size. Once it hits the max size, it removes the oldest / files not in use from the cache.

Is the implication here that --vfs-cache-max-size can take priority over --vfs-cache-max-age?

Thank you both for your help!

This gets super easy when you share a debug log with your question as we can just look through the log and show you what's going on.

In your setup, you have 100G max size.

You have max age 2 hours.

If two hours goes by and the files are not in use, they might get removed. If they are in use, they won't get removed.

If the size goes bigger than 100GB, it tries to remove the oldest not in use files first and if there are no, it removes nothing as it is all in use.

You see these entries in the log file.

2022/01/30 23:41:05 INFO  : uploadtest: vfs cache: removed cache file as Removing old cache file not in use
2022/01/30 23:41:05 DEBUG : uploadtest: vfs cache: removed metadata from cache as Removing old cache file not in use
2022/01/30 23:41:05 INFO  : vfs cache RemoveNotInUse (maxAge=10000000000, emptyOnly=false): item uploadtest was removed, freed 153 bytes
2022/01/30 23:41:05 INFO  : vfs cache: cleaned: objects 0 (was 1) in use 0, to upload 0, uploading 0, total size 0 (was 153)

If you want to make a debug log, it's much easier to show what's going on and see why something may get removed or not get removed.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.