Question re: caching in rclone mount; troubleshooting workflow

What is the problem you are having with rclone?

Question about caching in rclone mount.

Using a very circuitous route, I'm having Sonarr download via Putio, then using rclone mount to complete the "blackhole". Amazingly, it is working almost flawlessly. The one issue I couldn't figure out was why after leaving the nssm-generated rclonemount service running for a while, it wouldn't register the mount-side changes (i.e. when putio downloaded a new file).

I believe the issue was that I'd initially had --dir-cache-time at 72 hrs. I've changed this to 15 min and things seem to be working now.

My questions are two-fold: 1) is this stupid? 2) can I make it better?

Would very much any feedback on the command I'm using (below).

Btw, at first I did think to just schedule a recurring rclone move command, but mount seems to be working a little better at the moment. Would it be more efficient to just use rclone move? I'm worried Sonarr might start grabbing incomplete files during transfer.

I've read and tried to understand as much as I can:
rcloneorg/commands/rclone_mount/
rcloneorg/cache/#mount-and-dir-cache-time

What is your rclone version (output from rclone version)

rclone v1.50.0

  • os/arch: windows/amd64
  • go version: go1.13.3

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10

Which cloud storage system are you using? (eg Google Drive)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

mount putio: P: --allow-other --config "C:\Users***.config\rclone\rclone.conf" --log-level INFO --log-file "C:\rclone\Logs\rclone.log" --vfs-cache-mode writes --vfs-read-chunk-size 32M --vfs-read-chunk-size-limit off --dir-cache-time 15m --drive-chunk-size 64M

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

2019/11/12 22:24:39 INFO : Putio root '': poll-interval is not supported by this remote
2019/11/12 22:36:39 INFO : Cleaned the cache: objects 12 (was 12), total size 3.159G (was 0)
2019/11/12 22:37:39 INFO : Cleaned the cache: objects 12 (was 12), total size 3.376G (was 3.159G)
2019/11/12 22:38:39 INFO : Cleaned the cache: objects 12 (was 12), total size 3.593G (was 3.376G)
2019/11/12 22:39:39 INFO : Cleaned the cache: objects 12 (was 12), total size 3.812G (was 3.593G)
2019/11/12 22:40:39 INFO : Cleaned the cache: objects 12 (was 12), total size 4.027G (was 3.812G)
2019/11/12 22:40:46 INFO : TV/vidx.mkv: Multi-thread Copied (new)
2019/11/12 22:41:39 INFO : Cleaned the cache: objects 12 (was 12), total size 7.286G (was 4.027G)
2019/11/12 22:42:39 INFO : Cleaned the cache: objects 12 (was 12), total size 7.503G (was 7.286G)
2019/11/12 22:43:39 INFO : Cleaned the cache: objects 12 (was 12), total size 7.717G (was 7.503G)
2019/11/12 22:44:39 INFO : Cleaned the cache: objects 12 (was 12), total size 7.932G (was 7.717G)
2019/11/12 22:45:30 INFO : TV/blah/RARBG.txt: Copied (new)
2019/11/12 22:45:33 INFO : TV/blah/RARBG.txt: Copied (new)
2019/11/12 22:45:35 INFO : TV/blah.mkv: Multi-thread Copied (new)
2019/11/12 22:45:40 INFO : Cleaned the cache: objects 15 (was 15), total size 10.221G (was 7.932G)
2019/11/12 22:46:39 INFO : Cleaned the cache: objects 15 (was 15), total size 12.885G (was 10.221G)
2019/11/12 22:47:39 INFO : Cleaned the cache: objects 15 (was 15), total size 13.100G (was 12.885G)
2019/11/12 22:48:39 INFO : Cleaned the cache: objects 15 (was 15), total size 13.321G (was 13.100G)
2019/11/12 22:48:57 INFO : TV/videox.mkv: Multi-thread Copied (new)
2019/11/12 22:49:39 INFO : Cleaned the cache: objects 17 (was 17), total size 13.937G (was 13.321G)
2019/11/12 22:50:32 INFO : TV/videox.mkv: Multi-thread Copied (new)
2019/11/12 22:50:39 INFO : Cleaned the cache: objects 18 (was 18), total size 14.783G (was 13.937G)
2019/11/12 22:51:39 INFO : Cleaned the cache: objects 18 (was 18), total size 15.046G (was 14.783G)
2019/11/12 22:52:20 INFO : TV/***videoex.mkv: Multi-thread Copied (new)
2019/11/12 22:52:39 INFO : Cleaned the cache: objects 18 (was 18), total size 15.281G (was 15.046G)
2019/11/12 22:53:39 INFO : Cleaned the cache: objects 18 (was 18), total size 15.496G (was 15.281G)
2019/11/12 22:54:14 INFO : TV/***videoex.mkv: Multi-thread Copied (new)
2019/11/12 22:54:39 INFO : Cleaned the cache: objects 18 (was 18), total size 18.181G (was 15.496G)
2019/11/12 22:55:39 INFO : Cleaned the cache: objects 18 (was 18), total size 18.398G (was 18.181G)
2019/11/12 22:56:39 INFO : Cleaned the cache: objects 19 (was 19), total size 19.159G (was 18.398G)
2019/11/12 22:57:39 INFO : Cleaned the cache: objects 19 (was 19), total size 19.400G (was 19.159G)
2019/11/12 23:05:16 INFO : Putio root '': poll-interval is not supported by this remote
2019/11/12 23:05:16 INFO : Cleaned the cache: objects 15 (was 0), total size 19.448G (was 0)
2019/11/12 23:06:16 INFO : Cleaned the cache: objects 15 (was 15), total size 19.448G (was 19.448G)
2019/11/12 23:07:16 INFO : Cleaned the cache: objects 15 (was 15), total size 19.448G (was 19.448G)
2019/11/12 23:08:16 INFO : Cleaned the cache: objects 15 (was 15), total size 19.448G (was 19.448G)
2019/11/12 23:09:16 INFO : Cleaned the cache: objects 15 (was 15), total size 19.448G (was 19.448G)
2019/11/12 23:10:16 INFO : Cleaned the cache: objects 15 (was 15), total size 19.448G (was 19.448G)
2019/11/12 23:11:16 INFO : Cleaned the cache: objects 15 (was 15), total size 19.448G (was 19.448G)
2019/11/12 23:11:41 INFO : TV/sonarr_write_test.txt: Copied (new)
2019/11/12 23:11:41 INFO : TV/sonarr_write_test.txt: Removed from cache
2019/11/12 23:12:16 INFO : Cleaned the cache: objects 16 (was 16), total size 19.342G (was 19.448G)
2019/11/12 23:12:43 INFO : TV/***video.mkv: Multi-thread Copied (replaced existing)
2019/11/12 23:13:16 INFO : Cleaned the cache: objects 16 (was 16), total size 19.468G (was 19.342G)
2019/11/12 23:14:16 INFO : Cleaned the cache: objects 16 (was 16), total size 19.468G (was 19.468G)
2019/11/12 23:15:16 INFO : Cleaned the cache: objects 16 (was 16), total size 19.468G (was 19.468G)
2019/11/12 23:16:16 INFO : Cleaned the cache: objects 16 (was 16), total size 19.468G (was 19.468G)

Putio doesn't support change modification so we don't get notifications of changes which is why you need the directory cache lower.

Setting it to 15 mins means that it is more likely rclone will need to go out to putio to read directories. This may or may not be a problem for you. It doesn't sound unreasonable to me :slight_smile:

The other thing you could do is use the remote control and refresh the directory cache when you want to, eg set the directory cache time high and call vfs/forget when you know stuff has changed, or call vfs/refresh to fill up the entire directory cache manually.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.