4k plex streaming, how to buffer further ahead?

[enrico]

Hey, I have a 100mbps downstream and I'm trying to get a 4k movie to stream via plex. My testing movie bitrate is 49.9mbps so one would assume I could play it without much effort, however short buffering occurs every 3-4 seconds after playing. If I pause the movie for a 1-2 mins I can get about 5mins of playback before buffering starts again. I've scoured these forums for tips on cache settings but can't get much better performance. Here are my settings:

gdrive > cache > crypt

[gdrive]
type = drive
client_id = *** <--- this is set correctly
client_secret = *** <--- this is set correctly
scope = drive
token = ***

[secure-media-cache]
type = cache
remote = gdrive:secure-media
plex_url = http://127.0.0.1:32400
plex_username = ***
plex_password = ***

[secure-media]
type = crypt
remote = secure-media-cache:
filename_encryption = standard
directory_name_encryption = true
password = ***
password2 = ***

Mount script is:

rclone mount secure-media: /mnt/disks/secure-media \
--allow-other \
--dir-cache-time=160h \
--cache-chunk-size=16M \
--cache-chunk-total-size=50G \
--cache-info-age=168h \
--cache-workers=5 \
--cache-tmp-upload-path /mnt/user/appdata/Rclone/rclone_upload \
--cache-tmp-wait-time 60m \
--buffer-size 0M \
--umask 002 \
--rc \
--log-level DEBUG \
--log-file=/mnt/user/appdata/Rclone/secure-media.log \
--cache-tmp-upload-path=/mnt/user/appdata/Rclone/rclone_upload \
--cache-chunk-path=/mnt/user/appdata/Rclone/rc_chunks \
--cache-db-path=/mnt/user/appdata/Rclone/rc_db &

If I watch the network activity I can see the activity ramp up as soon as the file starts playing, when I pause the activity continues but only for a couple of mins. I would've expected that --cache-chunk-total-size=50G would mean the file would continue caching after pause. The activity only seems to hit ~45mbps despite my bandwidth being double that. Any ideas?

EDIT: I increased the chunk size to --cache-chunk-size=32M which slows the start time but seems to work much better. The traffic monitor shows the bandwidth getting much closer to my available downstream too, should I assume doubling this again to 64M would perform even better?:

[osatien]

i think that the download per file is just that 45MBs so you cant bypass that.

how can we help this?

[calisro (Rob)]

The newest beta has multithread download support under certain circumstances. I've not tested it but perhaps you could see if it helps.

Multi threaded downloads - comments and testers needed

This will work for anything which calls the internal operations.Copy function. This is sync/copy/move and friends copyto/moveto. If you are using rclone mount with --vfs-cache-mode full it will be active. It won’t be active for rclone serve though, but if you are using rclone serve http then you could use rclone to do multithreaded downloads from it.

You would need vfs in full mode though. I'm not sure how it works through the cache backend either.

[Animosity022]

Is there a particular reason you are using the cache backend? Try it without it.

What rclone version are using?
Are you using your own API key / Client ID?

https://rclone.org/drive/#making-your-own-client-id

When I start a high bitrate movie, you can see it will consume pretty much everything possible to play to get a buffer going.

[enrico]

Hi, thanks for your response. I've used a lot of your advice getting set up and am currently working through setting up your homescripts, they're very helpful and well written.

I'm on 1.47.0
Yes, own client ID.

Do you mean removing the cache remote like so:
gdrive > cache > crypt
to
gdrive > crypt
?

I'd read that can cause API bans. If this isn't the case then it'd make my life much easier to remove the cache altogether, however my understanding is that the cache specifically helped with media streaming.

Thanks, I'll read up on the multithread support.

[Animosity022]

Unless you are using a version from 2018, you'd be fine. The cache backend uses chunked reading, which is the default in rclone for some time now.

You are fine going from gdrive->crypt.

I'd try to that and see how your performance is and how the traffic looks.

[enrico]

Bah, I've spend countless hours reading advice from 2018. I'll stay more up to date from now on. Will report back once I've tested.

[Animosity022]

It's tough as it was released in May 2018 and information / guides tend to stagnant on the web usually so you'll find old guides/etc.

Even the term 'ban' isn't quite right as older version hit quota limits and such. No one was ever 'banned' from using rclone nor any other tools as there are daily quota limits.

[enrico]

I've had a few temporary holds put on my api key, although that's only been uploading, not downloading or streaming.

Does removing the cache remote affect my mount script at all?

[Animosity022]

I'd change it from your original script to something like:

rclone mount secure-media: /mnt/disks/secure-media \
--allow-other \
--dir-cache-time=160h \
--buffer-size 256M \
--umask 002 \
--rc \
--log-level INFO 

buffer-size is per file opened and heavily dependent on the memory on your server. I have 32GB of memory and plenty of spare so I keep a large buffer. YMMV.

[Animosity022]

And you'd have to change your rclone conf to point directly to your crypt:

[secure-media]
type = crypt
remote = gdrive:secure-media
filename_encryption = standard
directory_name_encryption = true
password = ***
password2 = ***

[enrico]

When testing today, changing --cache-chunk-size from ~16M to higher numbers made a huge difference. I assume this was just fixing problems caused by using the cache remote in the first place.

[Animosity022]

Yes, the cache workers x 5 is trying to keep up with a very high bitrate to play so making it bigger would help.

[enrico]

Works much more reliably on a slow connection. I'm changing my remote machine to test it with a 4k file.

[Animosity022]

The cache backend is another layer in between so it adds some latency in between.

There are use cases of when files are rapidly opened/closed that it helps but generally for playing with Plex, it's going to be a slower solution.

Regardless of a cache or non cache, if you don't have enough capacity to stream a higher bitrate movie/show due to a slow connection, nothing can fix that as you either have to limit in Plex to a lower bitrate or get a faster connection.

[enrico]

rclone mount secure-media: /mnt/disks/secure-media \ --allow-other \ --dir-cache-time=160h \ --buffer-size 256M \ --umask 002 \ --rc \ --log-level INFO

Just tested on the remote machine, looks like that's done the trick. Thanks heaps for your help. I'll keep more up to date on rclone development.

[Animosity022]

That graph seems more expected as it should be 'spiky' in terms of playing. Happy you got it working.