Rclone VFS Cache not clearing after Upload

What is the problem you are having with rclone?

Hi, if I am understanding correctly then VFS Cache should be empty once all the files/directories uploaded into the Google Drive.

But that's not the case and I can see storage space is still occupied by cache. I am sure I am doing something wrong :slight_smile:

Run the command 'rclone version' and share the full output of the command.

rclone v1.60.0-beta.6466.2aa264b33

  • os/version: alpine 3.16.2 (64 bit)
  • os/kernel: 4.15.0-193-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.19.1
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

  - RCLONE_CONFIG=/config/rclone.conf
  - RCLONE_DIR_CACHE_TIME=9999h
  - RCLONE_GID=1000
  - RCLONE_TIMEOUT=10m
  - RCLONE_UID=1000
  - RCLONE_UMASK=002
  - RCLONE_LOG_LEVEL=INFO
  - RCLONE_CACHE_DIR=/storage/.cache
  - RCLONE_VFS_CACHE_MODE=full
  - RCLONE_VFS_CACHE_MAX_SIZE=250G
  - RCLONE_VFS_WRITE_BACK=1h
  - RCLONE_VFS_CACHE_MAX_AGE=9999h
  - RCLONE_VFS_READ_AHEAD=1G
  - RCLONE_TPSLIMIT=10
  - RCLONE_TPSLIMIT_BURST=10

command: mount "gdrive:" "/storage/gdrive" --allow-other --allow-non-empty

The rclone config contents with secrets removed.

Paste config here

A log from the command with the -vv flag

2022/10/02 08:50:09 INFO  : vfs cache: cleaned: objects 645 (was 645) in use 0, to upload 0, uploading 0, total size 233.937Gi (was 233.937Gi)

Yes - it takes 1 hour before it uploads.

and it takes 1 hour by default to remove it from the cache.

--vfs-cache-max-age duration         Max age of objects in the cache (default 1h0m0s)

If you posted a debug log, it shows you the detail and why it wasn't removed from the cache.

Hi Hassan,

... and then the file then stays in the cache for minimum 9999 hours, unless you exceed 250GB.

Perhaps you want to use RCLONE_VFS_CACHE_MODE=writes?

What are you using the mount for and what made you select the current parameters?

Oops - Thanks @Ole as I swear I looked for that and didn’t see it. Too early here!

No problem @Animosity022 - I hadn't seen the write-back, so I would say we are equal. Good morning! :grin:

Thanks @Ole and @Animosity022 for your replies.

I never used writes so not exactly sure but at least I know what I was doing with the AGE

If I use writes then RCLONE_VFS_CACHE_MAX_AGE= and RCLONE_VFS_READ_AHEAD= not required?

Thanks again

You are welcome!

OK, then I think the quick and easy solution is to keep RCLONE_VFS_CACHE_MODE=full and reduce RCLONE_VFS_CACHE_MAX_AGE to something small e.g. 1h.

Yes, I believe so (not completely sure) - probably easiest to keep "full" with a small AGE to avoid issues - the disk usage will be similar (in most situations).

Thanks again Ole and much appreciated

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.