Behaviour of `--vfs-cache-max-age` with long uploads

I am looking into a separate issue and would like to understand the following workings of rclone to help me with that issue.

I know that --vfs-cache-max-age defines how long vfs cached objects/files will be kept on the local file system (in a subdirectory of the --cache-dir setting).

My question is how this interacts with long uploads. For example:

  • If I set --vfs-cache-max-age 10m and then move (not rclone move, file system move) a 100GB file to a GDrive mount that will take > 10 minutes to upload
  • I can see the full file moved to a local /--cache-dir/vfs/--volname/ subdirectory
  • Then I see my nic’s bandwidth starting to get used for uploading to gdrive.
  • But I know that --vfs-cache-max-age will be triggered before this upload completes.

So how does rclone handle the triggering of a cache purge upon the cached file aging beyond --vfs-cache-max-age? Because the move would still need the cached source file yet the cache would want to purge it.

Apologies in advance if this info is already in the docs, I looked but didn’t notice this info specifically.

Rclone keeps track of files that are in use in the cache and ignores them when cleaning the VFS cache. So in this case the upload should continue without rclone deleting the file.

1 Like

OK got it thx @ncw.

And I assume that after the file finishes uploading from the cache it would then be deleted from the cache (assuming again that --vfs-cache-max-age had expired on that file during the upload).

That sound right?

The timer is reset when the file stops being uploaded, so it will be eligible for deletion --vfs-cache-max-age after the upload has finished.

1 Like

OK that makes sense, thx.