Mac OSX Cache Full | Next Steps to Clear/ Modify Workflow?

Help and Support
1

What is the problem you are having with rclone?

It appears my disk and cache are full. I have a 500 GB HD that Disk Utility shows as full. I tried to find the cache location, but am unsure how to clear it out. All has been working well, and uploads to GDrive show up immediately on Emby. I am thinking I need to modify some cache variables, but don't want to lose data if I can help it! Most settings were borrowed from @Animosity022 and their github.

What is your rclone version (output from rclone version)

Version 1.55.1 (On OSX and pasted into /usr/bin)

Which OS you are using and how many bits (eg Windows 7, 64 bit)

OSX 11.5.1, 64 bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive via Google Workspace Enterprise Plus

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount gdrive-media-crypt: ~/gdrive-local --allow-other --fast-list --daemon --dir-cache-time 5000h --poll-interval 10s --umask 002 --vfs-cache-mode full --vfs-cache-max-size 400G --vfs-cache-max-age 5000h --vfs-read-ahead 2g -vv

The rclone config contents with secrets removed.

[putio]
type = putio
token = {"access_token":"redacted","expiry":"0001-01-01T00:00:00Z"}

[gdrive-media]
type = drive
client_id = redacted
client_secret = redacted
scope = drive
root_folder_id = redacted
token = {"access_token":"redacted","token_type":"Bearer","refresh_token":"reacted","expiry":"2021-08-13T12:44:50.884014-04:00"}

[gdrive-media-crypt]
type = crypt
remote = gdrive-media:/media
filename_encryption = standard
directory_name_encryption = true
password = redacted

A log from the command with the -vv flag

Locking failed. No space left on device. (This is the OSX error when I open terminal)
2

New command and log. I went to ~/Library/Caches and cleared out the rclone directories. Freed up space.

New command

rclone mount gdrive-media-crypt: ~/gdrive-local --allow-other --fast-list --daemon --dir-cache-time 24h --poll-interval 10s --umask 002 --vfs-cache-mode full --vfs-cache-max-size 100G --vfs-cache-max-age 24h --vfs-read-ahead 2G -vv

New log

2021/08/25 09:49:17 DEBUG : Using config file from "/Users/tstraub/.config/rclone/rclone.conf"
2021/08/25 09:49:17 DEBUG : rclone: Version "v1.55.1" starting with parameters ["rclone" "mount" "gdrive-media-crypt:" "/Users/tstraub/gdrive-local" "--allow-other" "--fast-list" "--daemon" "--dir-cache-time" "24h" "--poll-interval" "10s" "--umask" "002" "--vfs-cache-mode" "full" "--vfs-cache-max-size" "100G" "--vfs-cache-max-age" "24h" "--vfs-read-ahead" "2G" "-vv"]
2021/08/25 09:49:17 DEBUG : Creating backend with remote "gdrive-media-crypt:"
2021/08/25 09:49:17 DEBUG : Creating backend with remote "gdrive-media:/media"
2021/08/25 09:49:18 DEBUG : fs cache: renaming cache item "gdrive-media:/media" to be canonical "gdrive-media:media"
2021/08/25 09:49:18 DEBUG : rclone: Version "v1.55.1" finishing with parameters ["rclone" "mount" "gdrive-media-crypt:" "/Users/tstraub/gdrive-local" "--allow-other" "--fast-list" "--daemon" "--dir-cache-time" "24h" "--poll-interval" "10s" "--umask" "002" "--vfs-cache-mode" "full" "--vfs-cache-max-size" "100G" "--vfs-cache-max-age" "24h" "--vfs-read-ahead" "2G" "-vv"]
3

hi,
so i guess you got it working?

-fast-list does nothing on a mount

4

@asdffdsa I'm not sure whether or not its going to just keep filling up the cache located at ~/Library/Caches and I have to keep manually emptying? In theory, the --vfs-cache-max-size and --vfs-cache-max-age should fix the disk filling, no?

5

i also have a emby server pointing to a rclone mount.
i do not use any vfs flags for streaming.

not an expert that vfs flags. hopefully Animosity022 will stop by soon.
-vfs-cache-max-size is not a hard limit.
if a file is open or in use, rclone will not remove it.

6

Ah! Gotcha. That's what I am trying to avoid. Deleting data from ~/Library/Caches that has yet to be transferred, and the disks filling up. All else works well. Sonarr and Radar can see the mounts perfectly, direct transfer, etc. I just want to try to fine tune these flags as best I can.

Mounting GDrive on OSX as --read-only made it unusable with Sonarr and Radarr, but that seems to have been alleviated.

Thanks for the help! Hoping @Animosity022 can help fine tune this a tad. Worst that happens is I have to monitor it a bit and clear the cache folders manually.

7

The best thing to do would be to put just into INFO and validate what's being cleaned:

2021/08/25 14:35:58 INFO  : vfs cache: cleaned: objects 7118 (was 7130) in use 1, to upload 0, uploading 0, total size 749.979Gi (was 750.300Gi)

It can go ver your 'hard' limit in some situations, but not by a whole lot.

I have 1 TB disk that I just use 750GB for cache as the cache is overkill anyway.

8

What command did you run to get that output? I know that's probably a dumb question.

9

I run my mount with --log-file /opt/rclone/logs/rclone.log and it runs with --log-level INFO

10

Got it! I'll have to give it the 24 hours to make sure I don't overwrite or kill a transfer and re-run. Thank you!

11

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.