Rclone errors with Cache and VFS

Help and Support
1

About two months ago I made the switch from Plexdrive to rclone for caching. Since then I’ve ran into numerous issues.

I am running:
Ubuntu Server 16.04
rclone v1.44-029-g04a0da1f-beta

  • os/arch: linux/amd64
  • go version: go1.11.1

After doing some reading I saw that there were two different methods to use rclone, VFS and cache. As a result I setup two separate mounts to try them both to see which would work best for my needs. Below is the cache service followed by the VFS services below.

ExecStart=/usr/bin/rclone mount plexcache: /home/plex/rclone
–allow-other
–buffer-size 64M
–dir-cache-time 20m
–cache-chunk-total-size=12G
–cache-db-path=/dev/shm/rclone
–tpslimit=8
–cache-workers=5
–drive-chunk-size 64M
–log-level INFO
–log-file /home/plex/logs/rclone.log \

ExecStart=/usr/bin/rclone mount plexcache2: /home/plex/rclone2
–cache-tmp-wait-time 5m
–allow-other
–buffer-size 64M
–dir-cache-time 20m
–vfs-read-chunk-size 128M
–vfs-cache-max-age 20m
–vfs-read-chunk-size-limit=1G
–log-level INFO
–log-file /home/plex/logs/rclone2.log \

Using either mount I constantly see errors like this in the logs:

ReadFileHandle.Read error: EOF
(196608/7845247015) error (chunk not found 0) response
ReadFileHandle.Read error: low level retry 1/10: EOF
(196608/7845247015) error (chunk not found 0) response
ReadFileHandle.Read error: low level retry 2/10: EOF
(196608/7845247015) error (chunk not found 0) response
ReadFileHandle.Read error: low level retry 3/10: EOF
(196608/7845247015) error (chunk not found 0) response
ReadFileHandle.Read error: low level retry 4/10: EOF
(196608/7845247015) error (chunk not found 0) response
ReadFileHandle.Read error: low level retry 5/10: EOF
(196608/7845247015) error (chunk not found 0) response
ReadFileHandle.Read error: low level retry 6/10: EOF
(196608/7845247015) error (chunk not found 0) response
ReadFileHandle.Read error: low level retry 7/10: EOF
(196608/7845247015) error (chunk not found 0) response
ReadFileHandle.Read error: low level retry 8/10: EOF
(196608/7845247015) error (chunk not found 0) response
ReadFileHandle.Read error: low level retry 9/10: EOF
(196608/7845247015) error (chunk not found 0) response
ReadFileHandle.Read error: low level retry 10/10: EOF
(196608/7845247015) error (chunk not found 0) response
ReadFileHandle.Read error: EOF

After doing some reading on here on the forums I have saw that running the below command should correct those errors. That has not been the case for me.

rclone --cache-db-purge lsd plexcache:

The only thing that has stopped those errors temporarily is to reboot my server. Things start to work correctly for a little while but then the errors start appearing again.

I’m kinda stumped and not sure what to do or where to go from here so any suggestions or recommendations would be greatly appreciated.

2

Those errors look like cache backend errors. In your plexcache2: mount, have you made sure it isn’t using the cache?

3

I see those type of errors using cache or vfs. Plexcache2 is a cache of my plexdrive2 mount. Should I mount plexdrive2 instead of plexcache2 when using vfs?

4

I can’t quite figure out what you want to do from reading the thread.

Your errors are related to something removing the chunks from that were previously downloaded from the local disk so it tossed an error.

(196608/7845247015) error (chunk not found 0) response
ReadFileHandle.Read error: low level retry 1/10: EOF
(196608/7845247015) error (chunk not found 0) response
ReadFileHandle.Read error: low level retry 2/10: EOF
(196608/7845247015) error (chunk not found 0) response

The cache-purge deletes the cache.db file which is the listing of all the files / folders that is kept.

I don’t use cache at all for my plex use as the regular vfs works better / faster.

I’d personally ditch the cache config and just use the vfs (stock) backend for plex and see how that works.

5

I’m trying to figure out what is causing the errors in both cache and vfs. The only way to correct the issue is a reboot. Cache is using RAM to store the chunks and vfs is using my SSD.

6 
–cache-db-path=/dev/shm/rclone

That’s only the cache.db, which isn’t the chunks.

 --cache-chunk-path

Is where the downloaded chunks are so you’d have to fix that. If you want to use the cache backend, you’d want to set --buffer-size to 0M as it does its own thing with the chunks so you are double using memory if you set it to anything other than 0.

For VFS, it doesn’t store anything at all locally and only uses memory. Memory is used per file and configured by buffer-size. If you have 64M configured and have 10 files in use, you’d consume 640M of memory.

7

I’m going to correct my cache backend and see if that fixes it. I’ll make some changes to the vfs one and see which one I like better