Plex 4K streaming (direct stream and direct play)

Help and Support
1

Hello

I have been scanning across these lands all day and although my findings have been very informative. Nothing has cured my issue

I have a VPS that i use for Plex and use Rclone to mount Google Drive to it and when i am trying to stream 4K (via direct play or direct stream) it gets so far then buffers forever until Plex calls it quits

I have a feeling it could be Rclone not 'being man enough' for what Plex is trying to do but that is a bit of speculation in all honesty

i was wondering if anyone could point me in the correct path?

Judging on other discussion i assume you'd need the follow but of course, if you need anything else i'll sort it for you

Server: Ubuntu 18.04 LTS - Hetzner

Rclone config:

[GDrive]
type = drive
client_id = Custom one i made
client_secret = Secret that follows
scope = drive
token = TOKEN
root_folder_id =
chunk_size = 64M

[GCache]
type = cache
remote = GDrive:Plex
plex_url = 127.0.0.1
chunk_size = 64M
info_age = 1d
chunk_total_size = 10G

[plexlib]
type = drive
client_id = Custom Client ID i made
client_secret = Secret that follows
scope = drive.readonly
token = "Token"
chunk_size = 32M

Now the "plexlib" was more me testing, i tried to avoid using the cache entirely and just see what happened if i mounted the drive directly as read only and pointed my plex library to it. My theory was the cache just being another step in the chain slowing it down but it did not go to plan if anything made it worse

My Systemd.Service

For the plexlib one if was messing around with
[Unit]
Description=RClone Service
After=plexdrive.target network-online.target
Wants=network-online.target

[Service]
Type=simple
ExecStart=/usr/bin/rclone mount plexlib:Plex /mnt/plexlib
--allow-other
--dir-cache-time=24h
--poll-interval 15s
--buffer-size 64M
--log-level INFO
--log-file /var/log/plexlib.log
ExecStop=/usr/bin/sudo /usr/bin/fusermount -uz /mnt/plexlib
Restart=on-abort

[Install]
WantedBy=default.target

And for the GCache
[Unit]
Description=RClone Service
After=plexdrive.target network-online.target
Wants=network-online.target

[Service]
Type=simple
ExecStart=/usr/bin/rclone mount GCache: /mnt/gdrive
--allow-other
--dir-cache-time=240h
--cache-workers=5
--cache-db-purge
--umask 002
--rc
--buffer-size 64M
--cache-tmp-upload-path /mnt/rclone_cache_upload
--cache-tmp-wait-time 60m
--log-level INFO
--log-file /var/log/rclonemount.log
ExecStop=/usr/bin/sudo /usr/bin/fusermount -uz /mnt/gdrive
Restart=on-abort

[Install]
WantedBy=default.target

I did play around with the cache sizes and making cache directories and cache DB directories, using --fast-list, increasing the workers and a whole lot more but i feel like i am more tryagnosing than diagnosing at this point

If you need anything else let me know

2

The question template is helpful so then people can see the log and the version of rclone.

Rclone will definitely stream fast enough. I'd drop the cache as many can stream just fine without it. I'd investigate the network and vps resources and latency between you and your vps.

3

Sorry, first time posting but noted

The version info is:
Rclone v1.51.0

  • os/arch: linux/amd64
  • go version: go1.13.7

as for resource utilization the CPU sits around 40% when playing media from rclone, Plex isn't doing anything at all but might jump to about 10% every so often. the network from Hetzner is not limited in terms of throughput. The average when playing media is about 180mbps download (maximum, it spikes a lot) and about 10mbps upload.

As for my round trip i get about a 60ms latency (my home internet is 450mbps)

I use the cache for writing to GD wouldn't removing it prevent that ability using rclone?

What settings are generally recommended for 4K?

4

No. You don't need the cache backend. You can still write to drive. Enable vfs-cache-mode=writes. A debug log with -vv would help.

5

try:: https://drive.google.com/open?id=11StCpjQrHsJOc9IGIbz-ZzjnxVWxDa_7

That's the log for me playing a TV program using the plexlib configuration not the GCache/GDrive setup so essentially bypassing the cache entirely

6

Okay so an update to this

I took your words of wisdom and got rid of the cache. I made a new remote for the drive which is as follows:

[Google]
type = drive
client_id = one i made
client_secret = One i made
scope = drive
token = TOKEN
root_folder_id =

Here is my systemd.service for the new config:

[Unit]
Description=RClone Service
After=plexdrive.target network-online.target
Wants=network-online.target

[Service]
Type=simple
ExecStart=/usr/bin/rclone mount Google:Plex /mnt/gdrive
--allow-other
--dir-cache-time=240h
--cache-workers=7
--cache-db-purge
--umask 002
--rc
--buffer-size 128M
--vfs-cache-mode=writes
--cache-dir /mnt/rclone_cache_upload/
--log-level DEBUG
--log-file /var/log/googledrive.log
ExecStop=/usr/bin/sudo /usr/bin/fusermount -uz /mnt/gdrive
Restart=on-abort

[Install]
WantedBy=default.target

And a googledrive.log (597.2 KB)

Now i have seen some improvement. It buffers however does try and start playing again (this is via direct play not direct stream). I am not hitting the buffering wall so to speak where Plex just gives up and tell me to go away

7

Some good things in looking at your log.

The Plex Player isn't constantly opening/closing the file so that's a good thing. I can see just the normal 3 opens in the log.

Direct Play means the file does not go through the transcoder so you are limited to buffers on your player only so you are very much limited to any latency between the player and the server. Increasing buffer-size may help for this but maybe not.

As you noted, you see a lot of 'spikes' in how items are played as Plex requests chunks and it will use whatever it can to fill that buffer and repeat the process.

If you try Direct Stream, this does use the transcoder settings and just remuxes the container which has limited CPU overhead on it. You can adjust the plex transcoder settings to read ahead. If you use this and still get buffering, the problem is most likely from server to player and it isn't getting a consistent enough connection to play.

With my setup, I stream 4K without any issues but I am using a home server on a player I know is direct streaming on a network I have full control over that's gigabit with very good peering to my Google Drive on a hard wired 4K ATV.

As you toss in variables with wireless, VPSs and going from other providers, the game gets much harder to troubleshoot and you'll find it's tough to 'tune' your way out of network latency from a VPS provider.

8

Okay so i little insight to the setup i have. I have actually looked on your GitHub and posts which is where i found a lot of useful info however no cure :frowning:

Of course as above i mentioned i have the Hetzner server which is in Finnland (i'm UK based) so not a million miles away from me

the device i am testing on is a Samsung 4K TV with the Plex app built into it, the TV is wired in and i have VM 450mbps internet

My round trip has a 60ms response time to it, my opinion is that isn't too dramatic to stream with at all but of course if i am wrong i am wrong

With fear you saying go back and look at your GitHub you ******. What changes would you advise to make to rclone and if you are willing to provide.. Plex, i do understand this is not the Plex forums

I was tempted to increase the cache size on the remote to like 64mb and maybe the buffer size to say 256mb?

generally the content that goes via direct stream is because the audio needs transcoding but i can toggle a setting on my TV's Plex app to allow the audio as i can support it with the sound system but my goal is get both working as not all devices i own have support for it

9

Most TV apps has as super tiny buffer and would notice any blips in a constant connection. I tend to avoid all the TV built in Apps and would use a Shield/Roku 4K/ATV 4K.

Latency is one part of the equation, but any packet loss would also come into play. For some perspective, I have a 8ms ping to my resolved Google Drive API endpoint. Are you suing a US Google Drive or a European one? Anything to reduce latency down would help.

I thought you removed the cache? It's more a juggling act based on your specific setup. I have my setup documented that works very well for my very specific use case and I try to share what works for me. The problem with any guides is that your setup is so different from mine, it makes it tough to say exactly what would work for you. Some folks have had great success with using the cache backend. Some folks have had great success using plexdrive instead of rclone. Use what works for you and if something doesn't work, you have to experiment a bit. I used plexdrive for ages because it worked better. rclone works better for me now so I use that.

If it's direct streaming, you should be able to make this like 300 seconds or even higher. This means the transcoder will buffer up 5 minutes for me:

image

This does nothing when you are direct playing though as it's not using the transcoder, which is why I suggested making a bigger --buffer-size as that may help out as that means rclone will read ahead and keep that much in memory. The problem still means the VPS has to talk to the plex player so if there's latency / packet loss from that connection, nothing will help as if the buffer empties on the player, you see the buffering message.

10

Okay i will test on my TV using a third party device which supports plex, i have no idea what that will be just yet as it depends on what i got but i will come back to you with that

I had no idea you can choose a location for Google Drive so i don't know. I'd assume google are smart enough to notice i am UK based so would go for european but again, that's just an assumption. I have will 'have a google' and find out

Sorry yes i have removed the cache just my rclone vocabulary --- i meant Chunk size

Testing with the transcoder for direct stream i had that setting at 120 and i hit my so called buffering wall where Plex just gives up, i will have another play around with it again just for due diligence

I will increase the chunk size to 64M and buffer size to 256M then try again via direct play, i will also find another device and let you know. I'll clear the log down and start a fresh and further upload it. with packet loss i would expect to see other issues occur if that were the case not just direct stream/play for 4K. Everything else is flawless... again i could be wrong

I will do some testing and post an update with the results. Thanks for the guidance, much appreciated

11

Chunk size is just the range request for the HTTP call. You'll not see too much with changing that in general.

It depends on how sensitive things are to packet loss. Most things you'd probably not notice at all. If you were to lose something like 0.5% and you are pushing 100Mb/s, it would be very noticeable to something with a small buffer size. If you were only streaming 2-5Mb/s, you would never notice as the buffer would be there to mitigate it.

If you are only having a problem with 4K, it's a choke point from point A to point B, but it's figuring out if you can do anything about the choke point to fix it.

I'd assume it's how you signed up for the account in general. I use US for all my GSuite stuff and I can see I am very close to my end point APIs. 60ms to the endpoint doesn't seem close to me as it feels like it would be Europe to US or vice versa.

12

Okay so based on your advice i never changed the chunk size

Now packet loss i tried a simple test with ping using 65 bytes and a load of pings and didn't get any packet loss. Of course this is not 100% accurate as streaming workloads are very different but it shows it is responding. I might look to see what network monitoring utilities Ubuntu has that allow me to see it all

I tried via direct stream and direct play on my TV and i tried PS4 as that's all i have, i tried on my PC as well. I also can't transcode 4K down to 1080p but that's a different problem i'll address another day

I had changed the buffer size to 256M

Looking into google i could only find https://cloud.google.com/compute/docs/gcloud-compute#set_default_zone_and_region_in_your_local_client
Which i followed and contacted Google Support and they were about as much use as an ashtray on a motorbike.

My round trip to my vps floats very nicely around 55ms and 60ms. i don't think i can see the api endpoint for google it keeps asking me to pay for stuff

the results were

Direct stream: 11 seconds in then completely wrote its self off -both TV and PC
Direct Play: 5 seconds, buffer, 57 seconds, buffer and repeat every 10 seconds after -both TV and PC
PS4 - decided to transcode to 1080p (not got a 4K PS4) that just went very south very quickly

log: https://drive.google.com/open?id=1KnjSgdmCIak1g2NWcj4OFjOUH0ddGvdM

I think i may try Plex Drive just to see if there is any difference, if not then i may just use my old server i have at home (trying to get rid of it, why i moved to cloud) and mount the google drive on that just to eliminate connectivity issues from client to my VPS and that way it is purely down to rclone connecting to Google Drive

13

You can check from the VPS what you are connecting to and see what it looks like and where it is:

^Cfelix@gemini:~$ host www.googleapis.com
www.googleapis.com has address 172.217.12.202
www.googleapis.com has IPv6 address 2607:f8b0:4006:81b::200a
felix@gemini:~$ ping 172.217.12.202
PING 172.217.12.202 (172.217.12.202) 56(84) bytes of data.
64 bytes from 172.217.12.202: icmp_seq=1 ttl=56 time=7.90 ms
64 bytes from 172.217.12.202: icmp_seq=2 ttl=56 time=7.100 ms
64 bytes from 172.217.12.202: icmp_seq=3 ttl=56 time=9.41 ms
^C
--- 172.217.12.202 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 4ms
rtt min/avg/max/mdev = 7.901/8.435/9.408/0.696 ms

felix@gemini:~$ traceroute 172.217.12.202
traceroute to 172.217.12.202 (172.217.12.202), 30 hops max, 60 byte packets
 1  router.animosity.us (192.168.1.1)  0.183 ms  0.175 ms  0.154 ms
 2  * * *
 3  lga25s63-in-f10.1e100.net (172.217.12.202)  5.296 ms  5.286 ms  5.279 ms
 4  * * *
 5  * * *
 6  lga25s63-in-f10.1e100.net (172.217.12.202)  7.706 ms  7.527 ms  6.467 ms
 7  lga25s63-in-f10.1e100.net (172.217.12.202)  6.205 ms  7.401 ms  6.303 ms
 8  * * *
 9  lga25s63-in-f10.1e100.net (172.217.12.202)  10.954 ms  8.372 ms  9.088 ms
10  lga25s63-in-f10.1e100.net (172.217.12.202)  7.776 ms  8.200 ms  8.552 ms
11  lga25s63-in-f10.1e100.net (172.217.12.202)  8.786 ms  8.079 ms  8.027 ms

felix@gemini:~$ geoiplookup 172.217.12.202
GeoIP Country Edition: US, United States

That looks to be my endpoint for Google's API.

Yeah, that sounds like CPU bottleneck on the server as you need something pretty beefy to transcode 4k.

Packet loss/drops usually only hit on load so you'd have to really generate some load to see what's going on. I use netdata on my Linux boxes to track a whole bunch of things and that tends to be helpful for me.

In general if you are playing something, plexdrive does a better job as how it handles reading ahead and filling up it's buffer since it was designed a bit more for Plex (obviously :slight_smile: )It doesn't hurt to give it a try as it may work.

14

So update No. whatever i am on now....

I setup my Nginx reporting and could not see any packet loss, not really believing in the reliability of that i decided to address the latency first and come back to diagnosing packet loss later on

I have moved my VPS to Germany and this knocked it down to 30ms round trip...ish, quite a chunk taken away from my original 60ms. Everything back up an running i dived into the packet loss and used MTR and tested both Client>Server and Server>Client

I noticed i am getting packet loss of about 8% at my ISP level so i ran your billy basic speed test and noticed my original statement of 450mbps i pay for every month has dropped to 148-170mbps consistent with 3 tests 30 minutes apart. was not very impressed with that

Lastly which pretty much provides a conclusion to this. I moved one of my programs from Google drive the local SSD storage on my server. The ultimate test so to speak and although provided MASSIVE improvement to the issue, did not cure it. So although rclone and Plexdrive (i did try Plexdrive as well) are not helping they are not the root cause of my problem

I also noticed that this issue only seems to happen with content that requires more than about 20mbps throughput as other 4K content i have that runs about 25mbps works like a dream even with rclone in play, when i get higher than that it falls over as these programs i am testing want like 49mbps or thereabouts according to Plex

I may just bump up my server CPU spec and let it transcode to 20mbps and burn some datacenter down.

Looks like this lock down got my internet all messed up

15

Nice. Happy you made some progress as it always feels good to get closer.

If you have that capacity constraint, you can't really tune out of that as you've seen.

You need a beefy CPU to transcode 4K down so it's much better get 1080p content.

As an example of a movie, you can see some huge spikes as I have the bandwidth to spare most of the time so it'll grab everything it can to fill up the buffer (good thing for me).

image
image976×365 92.8 KB

I had to convince a friend that his 40-50Mb/s wifi would not do 4K effectively...

16

Yeah i think that is what the problem is here, the country is on lock down so everyone is at home and the ISP's cant handle it, (not knowing if you are UK based) when you buy an internet package they generally tell you what your are buying but provide "average speeds" they lower it based on demand. that seems to be what's happening to me here and i can imagine to many others

Yeah i only use a 2 vCPU server in all honesty as it works most of the time, it can transcode lower qualities and everything in 4K i use direct play for. i will just bump up make it 4 or 8.

Wow, your graph is a lot different from mine:

image
image975×793 230 KB

[EDIT] - That's playing from local media so no Rclone

17

Afternoon chaps.

I thought I might chime in as I am having similar issues streaming 4K as of the last week or so and my set up is incredibly similar to yours Julian.

  1. I'm in the UK
  2. I have a 450mbps Virgin fibre connection (however only seeing 240 down and 30 up at present)
  3. I have my Plex server on a Hetzner box
  4. 4k and also high bitrate 1080p is freezing with 'server too slow messages'

Although slower than expected, I don't think Virgin is the problem as 240mbps should be ample, but I am wondering if the Hetzner box could be the issue?

18

Potentially the Hetzner connection could well be an issue yes. Soon i am going to look to try a dedicated server instead of a cloud VPS with dedicated cores

if your VPS is using dedicated cores then you should find it ample within reason of course. As i say mine is 2 cores and works fine other than all of the above

try the MTR test and see if you are getting packet loss within the VM network. That's my current problem. It hits somewhere in Virgin then i have 8% loss which in the world of streaming 4K with a 30 or 60ms latency..... you got problems.

MTR Hetzner - http://wiki.hetzner.de/index.php/Netzwerkdiagnose_und_Report_an_Hetzner/en

Change the IP to your external home address

MTR for windows - http://sourceforge.net/projects/winmtr/

You can try client-server and server-client

19

Not sure if Hetzner has IPv6 but try binding your mount to your IPv4 address, Google sometimes limits peering with IPv6 and you end up doing a longer route, pretty sure Hetzner should have direct peering with Google so your ping time looks high in double digits.

Does your current ISP offer a speedtest.net server in your area? If so, try running a speedtest from your VPS server to your ISP server in your area, this will tell you the upload/download speed of your ISP and Hetzner in your area, it might be the downlink from your ISP is fast, but the uplink slow/congested, since it's the other way around you are getting limited.

Finally, if you have a computer to test, try mounting it in your home and do a test there. See if it buffers or not. For 4K the less trip your stream takes the better.

20

I have tried with moving the media onto the server its self so no rclone, no google etc... and the issue remains

the speedtest-cli does not have a VM server.. A general speed test from Hetzner comes in at ridiculous speeds but of course that doesn't mean much if it is not coming across to the UK