Cache-workers and plex integration

Help and Support
1

Hi,

I just upgraded to 1.4 and switched from gdrive -> plexdrive -> rclone/crypt mount to gdrive -> rclone cache -> rclone/crypt mount. At least I hope I have…

[GDRIVE]
type = drive
client_id = XXX
client_secret = XXX
token = XXX

[GDRIVECACHE]
type = cache
remote = GDRIVE:plex
chunk_size = 5M
info_age = 30m
chunk_total_size = 100G

[PLEXDRIVE]
type = crypt
remote = GDRIVECACHE:
filename_encryption = standard
password=
password2=

and then I

/usr/local/bin/rclone -v --cache-info-age=1h --cache-workers=4 mount PLEXDRIVE: /mnt/plexdrive --allow-other

I have two DSL 50Mbit lines which rclone can use both (via router round-robin) if it is using more than one worker. At first I was not using cache-workers. When I started direct playing a movie it immediately stuttered. It looked as if only one line was used (btw. it did not stutter with plexdrive). I assume rclone figured out that one movie was streaming and downgraded to one worker? Hence I removed the plex connection and switched it to four workers. No stuttering anymore and it looks as if both lines are used equally (which is a huge improvement compared to plexdrive).

Is this the optimal setup now (I am not sharing anything, just personal use)? Or are there more parameters to tweak to make things better and prevent bans? I am getting a few of these btw:

WriteFileHandle: Truncate: Can’t change size without --vfs-cache-mode >= writes

Thanks
JP

2

That means something is writing to the mount and trying to truncate things. To get round that message you’ll need the --vfs-cache-mode writes flag which will buffer writes locally before uploading them.

3

I use a temporary folder with the cache uploads to write anything to.

/usr/bin/rclone mount gmedia: /gmedia \
   --allow-other \
   --dir-cache-time=160h \
   --cache-chunk-size=10M \
   --cache-info-age=168h \
   --cache-workers=5 \
   --cache-tmp-upload-path /data/rclone_upload \
   --cache-tmp-wait-time 60m \
   --buffer-size 0M \
   --attr-timeout=1s \
   --syslog \
   --umask 002 \
   --rc \
   --log-level INFO.

Based on my config, it uploads to my GD in 60 minutes. I use the plex integration so that is bumps up the cache workers on playback. Start time for me is roughly 2 seconds to 4 seconds for any movie regardless of size, but I am running on gigabit FIOS. I’ve yet to see any stuttering once a movie starts.

My config is a self contained single machine so I have my Sonarr/Radarr/rTorrent/NZBGet/Plex all on the same server which is an i7-7700 with 32GB of memory. I have the intel quick sync HW transcoding configured as well so I barely break my CPU when it occurs and memory barely goes over 6-7GB in use.

4

Ok. But what should that be? Only plex is accessing this via a read-only access (FreeBSD jail). When I go into that jail and try to touch anything it correctly says “read-only filesystem”. So plex would not be able to write to that mount. However these messages start the moment I do something in Plex (e.g. watch a movie).

With --vfs-cache-mode writes enabled the messages are gone however something indeed seems to write to the mount as it then mentions “SOMETHING.mkv Copied (new)” so rclone tries to do SOMETHING. Maybe atime?

5

Indeed in the log I see things like

updateTime: setting atime to 2018-04-03 19:45:30.102402 +0200 CEST

which seems to be the root cause. Accessing the files also takes ages now with --vfs-cache-mode writes probably because rclone tries to change the atime now. I mounted with --no-modtime but still no change.

Moreover:

2018/04/03 19:49:14 DEBUG : Video/TEST.mkv: Setattr: a=Setattr [ID=0x9a86 Node=0x4 Uid=972 Gid=972 Pid=72454] size=2214979947 handle=INVALID-0x0
2018/04/03 19:49:14 DEBUG : Video/TEST.mkv: Truncating file
2018/04/03 19:49:14 DEBUG : Video/TEST.mkv: Open: flags=O_WRONLY
2018/04/03 19:49:14 DEBUG : Video/TEST.mkv: >Open: fd=Video/TEST.mkv (rw), err=
2018/04/03 19:49:18 DEBUG : : Sending chunk 192937984 length 8388608
2018/04/03 19:49:24 DEBUG : /: Attr:

all a bit spooky.

Followed by tons of “Sending chunk 763363328 length 8388608” messages (with different chunks). Why is rclone mount trying to truncate anything or change file size for a MKV file just because PLEX is reading (!) the file?