I have been troubleshooting plex playback issues for a couple weeks. Plex playback involves lots of pauses and slow starts. I have seen the same performance when using plexdrive or rclone cache.
Using rclone copy I can move data to and from googledrive usually between 12-25 MB/s
[root@ssdnodes-87861 Moana (2016)]# rclone copy gd-enc: /opt/docker/sabnzbd/downloads/complete/ --include "/Movies/Moana\ \(2016\)/**" -vu
2018/03/04 11:31:26 INFO : Local file system at /opt/docker/sabnzbd/downloads/complete: Modify window is 1ms
2018/03/04 11:31:27 INFO : Local file system at /opt/docker/sabnzbd/downloads/complete: Waiting for checks to finish
2018/03/04 11:31:27 INFO : Local file system at /opt/docker/sabnzbd/downloads/complete: Waiting for transfers to finish
2018/03/04 11:32:26 INFO :
Transferred: 1.348 GBytes (22.612 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 1m1s
Transferring:
* ... (2016)/Moana (2016) Bluray-1080p.mkv: 10% /13.418G, 18.562M/s, 11m5s
2018/03/04 11:33:26 INFO :
Transferred: 2.925 GBytes (24.743 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 2m1s
Transferring:
* ... (2016)/Moana (2016) Bluray-1080p.mkv: 21% /13.418G, 25.216M/s, 7m6s
Using rsync to measure speeds while copying from an rclone mount I see speeds in 1.2-5.5MB/s
I am using rclone encryption. The system has 16GB RAM and 2 virtual CPUs. I have seen similar performance on different cloud provides (ssdnodes, chicagovps, digital ocean). The VMs do not seem resource constrained. The size of the google drive mount is about 11TB.
Could you try a test without the cache, just with the mount and crypt? It would be interesting to see if it is the cache layer or the mount layer which is causing the problem.
I gave it another go with latest rclone rclone v1.39-246-g078d705dβ with using the rclone cache drive I experienced slightly worse results than I previously reported.
The results using rclone copy appear about the same
root@ssdnodes-87861 Moana (2016)]# rclone copy gd-enc: /opt/docker/sabnzbd/downloads/complete/ --include "/Movies/Moana\ \(2016\)/**" -vu
2018/03/17 11:21:00 INFO : Local file system at /opt/docker/sabnzbd/downloads/complete: Modify window is 1ms
2018/03/17 11:21:01 INFO : Local file system at /opt/docker/sabnzbd/downloads/complete: Waiting for checks to finish
2018/03/17 11:21:01 INFO : Local file system at /opt/docker/sabnzbd/downloads/complete: Waiting for transfers to finish
2018/03/17 11:22:00 INFO :
Transferred: 1.282 GBytes (21.673 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 1m0.5s
Transferring:
* ...ana (2016)/Moana (2016) Bluray-1080p.mkv: 9% /13.418G, 23.036M/s, 8m59s
[root@ec2-107-23-146-18 Moana (2016)]# rclone copy gd-enc: /tmp --include "/Movies/Moana\ \(2016\)/**" -vu
2018/03/17 16:05:36 INFO : Local file system at /tmp: Modify window is 1ms
2018/03/17 16:05:37 INFO : Local file system at /tmp: Waiting for checks to finish
2018/03/17 16:05:37 INFO : Local file system at /tmp: Waiting for transfers to finish
2018/03/17 16:06:36 INFO :
Transferred: 8.083 GBytes (136.303 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 1m0.7s
Transferring:
* ... (2016)/Moana (2016) Bluray-1080p.mkv: 60% /13.418G, 130.251M/s, 41s
I know speedtest aren’t everything, but it doesn’t look like rclone slowness is a network speed issue.
However, I was reading through another post on the rclone forum as saw this post indicating that if plex integration is setup, the stream will start with a single worker until its confirmed that plex has started playing. I have plex integration setup in my config. Maybe it was not working right and was limiting rclone to a single worker. When I removed the plex config from the rclone config I saw these speeds.
I’m not OP but I find similar result. Rclone mount with cache was around 10%-30% slower transfer speed for regular plex scans, analyzing, and playback. ‘rclone move’ via cache (not using mount) was about the same slower.
Disabling the plex integration via ‘rclone config’ gave me full speed for plex stuff, but bog down my minimalist 10watt cpu server with only 4gb RAM. Rclone would take up all the RAM and 2GB swaps for any plex scanning/analysis. I had to play a bit with the rclone options to get it the way i want. But ‘rclone move’ via cache was still around 30% slower compared to direct rclone move to GDrive:
In the latest beta, everything feels speedier without plex integration in rclone mount cache, so i left my old config just the same. Plex scans/analysis and playback is much faster but every few hours plex scans/analysis would do nothing like it was waiting for rclone to finish downloading chunks on a file, but rclone doesn’t download and doesn’t do anything.
My current workaround is to restart rclone cache mount with systemctl and right after that rclone would start downloading chunks again as plex scanner requested.
‘rclone move’ via cache also achieve the same speed as rclone move directly to GDrive. So thats solved.
When i enable plex integration in rclone config, plex scanning and analyzing is just way too slow that it never finish. Although to be fair i have rather big library around 45,000 files of movies and tv shows.
I believe it is written in rclone.org website that with plex integration rclone limit the number or thread or process it runs.
Edit: I imagine if remusb handles the parent of a new directory being missing cache issue then my other feature request will simplify, that being said, I guess it’d be handy to check for validity and still hard purge the parent if it’s missing just for sanity/making sure it imports.
I don’t have problem with plex auto scan for my big gdrive library with the latest stable 1.4 with plex integration disabled. Changes are immediately detected on plex whether it’s local or in gdrive, and i usually do a lot of changes in a day : renaming tv shows files, adding subs, uploading 25 movies, 50 episodes, etc all typically happen in a day. With api polling works, all of this are instantly picked by plex regardless i change it via cache or via gdrive webui etc.
However as soon as i use plex integration then plex scanning seems slow or stalled in a plex library section that’s why i left plex integration disabled. For example, plex would instantly recognize changes of all 50 episodes I recently uploaded in the past hour but plex scans would never finish tv shows section thus the 25 movies i just uploaded is never picked up. So with plex integration enabled, in this case i would have to cancel the tv shows section scan, and then initiate scan in movies section.
I tested with beta below, i enabled plex integration and plex scanning does seem to be much faster and so far not stalled in one plex section.
However i found two possible issues:
on plex playback, the initial buffer is longer than before. And sometimes it looks like it finished downloading buffer but plex won’t play. The only remedy is to close the player and retry again.
This seems to be similar bug in the past that reoccur again in this beta?
Basically if it buffers ok, plex will play immediately. If it fails to buffer initially, then even when the buffer fully downloaded plex won’t play the file. I checked via netdata when playing file.
So the initial buffer status somehow made big difference on plex playback. I’m guessing rclone is a bit slow on verifying actual plex playback as opposed to plex scanning?
i cant delete file from plex anymore. When i delete file via plex, say a movie, and i delete via plex web UI or plex android, it looks like its deleted but rclone doesn’t actually delete the file. So on plex service restart plex will find the same file.
Mar 28 12:20:21 jupiter systemd[1]: Started Mount and cache Google drive to /mnt/Plexdrive.
Mar 28 12:25:30 jupiter rclone[23134]: 2018/03/28 12:25:30 ERROR : tv/Foyle’s War/Foyles War S07/Foyles War S07E01 The Enternity Ring x264 RB58.mp4: File.Remove file error: googleapi: Error 403: The user does not have sufficient permiss
Mar 28 12:34:09 jupiter rclone[23134]: 2018/03/28 12:34:09 ERROR : tv/Rosemary and Thyme/Rosemary & Thyme 2x06 The Gongoozlers.mp4: unexpected conditions during reading. current position: 303140864, current chunk position: 293601280, cu
Mar 28 12:34:09 jupiter rclone[23134]: 2018/03/28 12:34:09 ERROR : tv/Rosemary and Thyme/Rosemary & Thyme 2x06 The Gongoozlers.mp4: (303140864/303142847) error (unexpected EOF) response
Mar 28 12:34:09 jupiter rclone[23134]: 2018/03/28 12:34:09 ERROR : tv/Rosemary and Thyme/Rosemary & Thyme 2x06 The Gongoozlers.mp4: ReadFileHandle.Read error: low level retry 1/10: EOF
lines 1-16/16 (END)
I shouldn’t have permission error like above. Prior to rclone 1.40 i could delete files via plex ok.