Rclone mount cache directory filling up and not being handled gracefully

What is the problem you are having with rclone?

using the latest rclone versions I keep on seeing my cache directory (really an oracle f80 ssd pci-e device) filling up and then rclone mount failing with errors

Filesystem      Size  Used Avail Use% Mounted on
/dev/sdc1       734G  697G     0 100% /home/spotter/rclone-cache

and it doesn't work again till I manually delete data

2022/08/29 19:06:58 ERROR : <snipped>: vfs cache: failed to _ensure cache vfs reader: failed to write to cache file: write /home/spotter/rclone-cache/vfs/<snipped>: no space left on device

if the device fills up, rclone should handle it gracefully. In this case, the problem is due to me forgetting to -m 0 after I reformatted device with ext2 so it had less space than I thought so rclone thought it had more space available to it then it did. but instead of just saying "woops I'm out of space, let me reclaim some" it fails in a terrible way.

it should handle this gracefully, it is a configuration error in a sense, so perhaps if it thinks it should have more space than it actually does it should log that fact.

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.1

  • os/version: ubuntu 22.04 (64 bit)
  • os/kernel: 5.15.0-46-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.18.5
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

google drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

it's very possible that there are bad things here (thought it used to work fine)

 rclone mount --cache-dir /home/spotter/rclone-cache --allow-other --no-modtime --attr-timeout 8700h --stats 10s --buffer-size 16M --vfs-cache-poll-interval 30s --poll-interval 30s --dir-cache-time 8700h --vfs-cache-max-age 96h --async-read=true --vfs-read-wait 30ms --vfs-write-wait 30s --vfs-cache-mode full --vfs-read-chunk-size 16M --vfs-read-chunk-size-limit 1G --local-no-check-updated --drive-chunk-size=64M --multi-thread-streams=4 --multi-thread-cutoff 250M --transfers 8 --drive-disable-http2=true --fast-list --vfs-case-insensitive --vfs-cache-max-size 0.7T --log-level ERROR --log-file=/tmp/rc-mount.log --rc gcrypt: /data/gdrive

The rclone config contents with secrets removed.

not at issue here

A log from the command with the -vv flag

pasted snippet above

hi,

a debug log would show what is going on.

if a file is in-use/open/locked, than rclone cannot purge it from the vfs file cache.

--fast-list does nothing on a rclone mount

There is an issue for this:

When disk full errors cause operation to terminate, VFS cache doesn't clear · Issue #4442 · rclone/rclone (github.com)

I personally don't agree with it (just my own opinion) as filled disks break everything generally. I put monitoring around it and ensure the space doesn't run out.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.