Major (random) performance issues (Rclone, Google Drive, Plex)

Help and Support
1

Hey All,

So I recently built a new Plex server, and ever since it's been running, performance hasn't been right at all. I can't for the life of me figure out what the problem is. Here's what happens: SOMETIMES, people and apps (Samsung TV's, Roku TV's, android box, etc.) can connect fine, and browse menus fine, and load movies fine, like it might be OK for an hour, or a few hours with no problems. Other times it takes forever to load a movie (like 15-30 seconds), or even just 10-20 seconds just to bring up the info screen for a movie, or it might just throw the error "can't connect to server, please check your internet connection". This error might be there for a few seconds, to a few minutes, and then all of a sudden it just reconnects.

I had a previous server running for over a year using rclone and Google drive without a problem. Menus loaded fast, movies started almost instantly, as would be expected over gigabit internet.

My mount command is super simple:

mount --log-level DEBUG --log-file C:\rclone\log.txt GDrive: x: --config "C:\Users\Admin.config\rclone\rclone.conf"

So I've left everything at default, which was working fine previously on my old server. I have a separate API key setup just for the rclone mount as per Annimosity22 settings.

I'm on wired Gigabit (up and down) fiber internet, server is an i3-xxxx with 16 GB memory and all SSD storage. Speedtest benches 900+ mbps consistently, at any time of day. I've stress tested the server for memory problems, cpu problems, network connectivity problems, etc., nothing seems off, just Plex.

my config file as follows:

[GDrive]
type = drive
client_id = ***********
client_secret = ***********
scope = drive
root_folder_id = ***************
token = **********************

Windows 10 pro, latest edition, rclone 1.53.2, latest plex server.

I changed rclones log file to DEBUG, as info wasn't throwing any errors. I've attached it as it's huge. log.txt (4.0 MB)
EDIT: There are some errors right at the beginning, but I have no idea what they mean...

2

Those errors at the front are important, they boil down to this

dial tcp: lookup www.googleapis.com: no such host

Which means that your DNS said the domain www.googleapis.com didn't exist, which is bad and probably the cause of your problems.

So I think your local DNS is broken... Where are you getting it from? You could try Google on 8.8.8.8 or cloudflare on 1.1.1.1?

3

Hi Nick,
Thanks for the reply.
I am in fact using cloudflare / google DNS, 1.1.1.1 primary, and 8.8.8.8 secondary.
I tried switching them, so Google is now primary, results seem the same, for some reason it's still showing no such host in the log file.
I just updated all drivers manually as well, incase something like the ethernet driver from Windows wasn't correct, same result.
This problem is intermittent as I described in my first post. I guess the rclone log showing unable to resolve googleapis.com would explain why sometimes Plex is unreachable, but what could cause this to happen intermittently?

I reset my client secret just for fun. updated drivers just in case. And also factory reset my modem, setting google 8.8.8.8 as primary DNS, and cloudflare as secondary. Problem seems the same...
No other machines on the network have any connectivity issues to Google or other services.

Uploaded a new log...
log.txt (687.1 KB)

4

Those are network issues you are having on that computer/client.

2020/10/29 09:42:09 INFO  : Google drive root '': Failed to get StartPageToken: Get "https://www.googleapis.com/drive/v3/changes/startPageToken?alt=json&prettyPrint=false&supportsAllDrives=true": dial tcp: lookup www.googleapis.com: no such host

You'd want to figure that out as you might have AV/security/something blocking your connection or DNS.

5

Thanks for the replies!

I reset firewall settings in Windows Firewall to default... And set the GDrive service at delayed started, to give windows a chance to "settle" before starting the service (I dunno, just thought it might make a difference).

I know don't get the "errors" in the log with regards to being unable to resolve googleapis.com, however, the same problem of menus / files taking a long time to load persists.

I rebooted the server, let it sit for 15 - 20 mins, tried to open a movie, and while it did open, it took about 20-30 seconds. I believe in the log, the entire time from 12:58:59 - 12:59:20 correlates to trying to play the movie file.

log.txt (828.2 KB)

6

Looks like your player is opening and closing the file repeatedly, which would cause delays. Probably best to check out the new vfs-cache-mode full.

7

Thanks, I'll read about and enable this.
Previously I believe you had to create a separate cache remote, is this still the case? Or do I just add this to the mount command and specify a location?

8

It's not a separate backend, but part of the standard rclone so you'd just add.

9

Thanks for the replies from both of you!

I've enabled vfs-cache, and unfortunately the exact same thing is happening... I see no difference cache enabled vs no cache. Again, sometimes files load instantly (under 2-3 seconds) as they did on my previous server, and sometimes there's a long pause before loading.

I've attached a new log. If you skip down to 2020/10/30 08:59:07, you can see the 21 second pause between clicking on movie, and the movie playing. It's almost like clicking "play" freezes rclone temporarily.

Any ideas?log.txt (1.4 MB)

10

you can run a continous ping with timestamp.
when you notice that 21 second pause, look at the ping history.
it might show something of interest.

this is from one of the greatest collection of free portable app collections, nirsoft

image
image783×539 94 KB

11

Thanks! That's a cool tool, unfortunately, it just shows my connection is rock solid, and rclone isn't functioning properly :joy:

I just tried opening a file... It took exactly 45 seconds to play the movie while the constant pings were being returned in 2 ms.

I also downloaded an Ubuntu ISO right afterwards, which went at a consistent 90-100 MB / s.

EDIT: I can also open the mount in Windows explorer, and download files fast and consistently...

log.txt (2.3 KB)

ping
ping1135×1030 237 KB

12

yeah, nirsoft tools are cool.

in that latest log file, there is no errors.
pehaps the delay is from
Re-reading directory which took 28 seconds.

can you post you latest mount command.

13

Previously, on my old server, mount command was:

mount --log-level INFO --log-file C:\rclone\log.txt GDrive: x: --config "C:\Users\Admin.config\rclone\rclone.conf"

The above was working fine for over a year without any connection problems.

Currently, as per suggestions above to enable vfs-cache, it's currently:

mount --vfs-cache-mode full --cache-dir D:\ --log-level DEBUG --log-file C:\rclone\log.txt GDrive: x: --config "C:\Users\Admin.config\rclone\rclone.conf"

D:\ is an empty ssd doing nothing else.

14

i would read this as this is what i do. from @VBB
Plex playing, buffering, stoppage - rclone mount issues

for mounts for media streaming, i use a read-only mount and use priming when i add media.
gdive does polling so no need to continually update the dir cache.

tl;dr
if you want you can add --vfs-cache-mode full --cache-dir D:\ to the command below

I suggest you try my mount settings. Note that this is a read-only mount. Create a .bat file with the following
(replace my path and drive name with your own, of course):

@echo off
title Rclone Mount READ ONLY
D:\Programs\Rclone\rclone mount --buffer-size 256M --dir-cache-time 1000h --poll-interval 15s --rc --read-only --timeout 1h -v Google_Drive_Crypt: G:
pause

Then create another batch file with this (replacing my path again):

@echo off
title Rclone Prime
D:\Programs\Rclone\rclone rc vfs/refresh recursive=true --fast-list -v
pause

Run this after you mount the drive to "prime" it. This will basically cache the file and folder structure for much, much faster reading afterwards, and it does so very quickly. You'll want to re-run this whenever you make changes to your mount.

also, depending on which cloud service you are using this can be very helpful and i use it.
--no-checksum --no-modtime

1 Like
15

Thanks! I will try this and report back. Cheers!

16

@Mark017

Based on your description of the issues you are experiencing, I would begin by troubleshooting without RCLONE involved. Things like the info screen delays do not have anything to do with rclone. That tells me there is certainly something else going on other than an issue with RCLONE. Certainly, the rclone logs say you have some network issues going on. I would disconnect rclone, put some media on a local hard disk, and make sure that media worked perfectly (my guess is you will still have problems you will need to figure out, but at least you are narrowing the issue).

Beyond just making sure your network connection is 100% (which it does not appear to be), you may also want to look for issues with the local plex database. I've had to spend a lot of time dealing with issues that stem from concurrency and locking problems when using the local SQLite database. Certainly if there is a local hard disk issue, a large database, a wrong internal SQLite setting, etc you are going to have a ton of issues.

Remember, for every action you take with Plex, there are TONS of interactions with the local database (time tracking, transcoding status, play status, play counts, file locations, etc.).

Just food for thought.

18

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.