I have Google Drive mounted with rclone, but it seems that Google Drive is throttling me sometimes. Not quite sure what causes it.
Essentially, plex playback is fine for almost all files and can reach 500+ mbps during playback. Rarely, some files will not playback correctly. I will see traffic that starts at 100 mbps that drops down to 10 mbps. I also noticed that copying a file from the rclone mount will only result in 10 mbps transfer every time regardless of the file. Downloading any file directly from the web interface in Chrome also results in a measly 10 mbps for any file, even those shared by other users.
Hereâs the strange thing there are three ways to increase the download speed.
If I begin playback of another file in Plex while copying a file from the mount, the copy speed on another computer immediately increases to 350 + mbps. Even stranger, the playback of a file also affects the download speed in Chrome as well. The Chrome download speeds also increase to 30MB/s. Stopping playback in Plex also results in the speeds decreasing again.
If I download any Google Drive file remotely using a VPN (locally hosted), it will increase the download speeds on my local network. Downloading a file from two different PCs locally or two files on the same PC make no difference in download speed.
Using a public VPN resolves download speeds.
Here are my RClone settings:
âallow-other --buffer-size=2048M --drive-chunk-size=256M --vfs-read-chunk-size=256M --vfs-read-chunk-size-limit=0 --transfers=10 --rc --tpslimit=5 --timeout=1h -vv --dir-cache-time=72h --poll-interval=55s --vfs-cache-poll-interval=1m0s --vfs-cache-max-age=73h
I run RClone in docker using this container that I forked.
TLDR: Google Drive is acting weird and can speed up downloads randomly. Playback speeds up file transfers.
2G of buffer means that it will read up 2GB for every file you touch and use 2GB of memory per file. Lots of waste there.
transfers does nothing on a mount so it should be removed
vfs-cache-poll and max age are useless since you arenât using any vfs-cache mode.
not sure why they changed the poll interval from 1 m to 55 seconds. Seems just odd.
If you have a specific playback problem with debug log to look at, weâre happy to help. Please post those details.
Thanks for the reply. I have read a lot of your posts in the past and based a lot of my settings off your original vfs settings. I noticed that you adjusted them recently but havenât changed my settings to match because they had been working fine with playback.
Yes. I have my own API key.
I changed the poll interval because I assumed it had to be smaller than the vfs-cache-poll, but if they are doing nothing, Iâll remove it.
I donât have any playback problems with Plex really. Everything works smoothly for playback. Its just copying (or downloading a file) that I have issues with. Something makes me feel that Plex has some sort of âmagic readingâ that speeds up everything elseâŚ
The mount is shared using samba in docker. I am using a windows device on the network to copy files down. It copies at slow speeds of 1MB/s and speeds increase when I playback a file in Plex.
Also Iâll add that running the optimize files option in Plex results in slow download speeds as well. These speeds also increase when playing back a file. It seems that there is some specific read method that is required to achieve the full download speed?
Think of vfs-read-chunk-size not as a buffer, but how it requests some the provider. It asks for âchunksâ of that and you still download those chunks. I leave all of that as the defaults.
Thatâs these two:
--vfs-read-chunk-size SizeSuffix Read the source objects in chunks. (default 128M)
--vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached. 'off' is unlimited. (default off)
Hmm thanks. I donât believe I have to debug samba thankfully though. It has something to do with Google Drive and how RClone requests a file? Running a cp from the mount to another directory results in the same problem.
Its baffling that the downloads from Google Drive in Chrome have the same problems which can be âresolvedâ by playing a file in Plex?? I really just want to get to the bottom of why any other type of file transfer beside a Plex playback is slow from both the mount and Chrome.
By buffer drops, do you mean it releases the memory? I have 32GB of memory allocated for RClone so I wasnât terribly worried about memory consumption as long as it improves performance. If its not doing anything performance wise, I would want to remove it though.
I created a log that includes annotated pictures of the file transfers.
The log shows:
Played back readfile.mp4 in plex which resulted in high download speed 500 mbps. Scrubbed to different location. Closed file.
Begin copying the copyfile.mp4 at a slow 2MB/s. Start playback in Plex which speeds up the copy on a different machine. Playback eventually finishes buffering to device and stops requesting data. Copy slows down again. Scrub to new location in Plex to force it to download again. Copy speeds up again.
I also included screenshots of the file transfer and the traffic from #2.
You have buffer size twice.
checkers does nothing on a mount so can be removed.
Why would you tpslimit it? not sure why that is there.
allow-non-emtpy is just an awful parameter as it allows over mounting and masks things. Just not a good one to ever use imo.
Can you just run the copy by itself and stop it after a few and letâs look at that.
Thanks for catching the duplicates. I removed them, but it made no difference with copying.
In the first log, I used windows to copy from the samba share. This time, I used the same rsync command you used in your screenshot from the machine its mounted on with the same miserable speeds.
The tpslimit was because I was getting API errors in the console when Plex did scans.
Scans shouldnât produce any API issues. Was that before or after using your own key? I tested many times and scanned 30-40TB in a few days without any API issues.
I had used rclone since before there was a built in key so I always had my own key. I think it might have been the plex scheduled tasks that happen at night. I would check the api console and there would be errors. The tpslimit got rid of them. Additionally, there were errors that would appear infrequently until I added the tpslimit. Regardless though, removing the tpslimit resulted in the same speeds.
The OS is rancheros. In the logs that I have been running, they have been going through mergerfs. But when you mentioned it, I just tried running it without mergerfs directly but had the same results.
At this point, it really does seem like Google is throttling me, but I am not sure why the speeds would increase when playing back mediaâŚ
Also, I would like to add that downloading a file in Chrome has the same effect. And a public VPN will fix the issue when I download a file from Chrome.
Iâm quite sure google isnât aware you or exist in terms of our little consumption
You got a lot of layers going on with dockers/mergerfs and quite some things so itâs more likely itâs something along those lines causing an issue.
If you are using mergerfs, you want to make sure you are using sync_read, but I didnât see anything out of order in your logs.
Also, downloading via the web uses a completely different way to get files as most people generally report that being fast and rclone being slow as rclone uses drive API and the download does not.
If both are slow, Iâd check router/ISP/machine/bad cable/etc.
Just to confirm, I am using mergerfs with sync_read.
The reason I bring up the downloads with Chrome is because the plex playback with Rclone also affects the download speeds through Chrome. I get significantly faster speeds when playing back media in Plex. The slow speeds ~1MB/s I get in Chrome also match up to the same slow speeds ~1MB/s I get in Rclone when copyingâŚ
I feel like I would have blamed the router or ISP quickly if the speeds were always the same. It just really doesnât make sense that this is happening.
I also have a ubuntu server machine that I originally had rclone using cache with these settings:
/usr/bin/rclone mount plex: /media/plex
âallow-non-empty
âallow-other
âlocal-no-check-updated
âbuffer-size=0
âcache-chunk-path="/media/cache/chunkcache"
âcache-chunk-size=8M
âcache-db-path="/media/cache/cache.db"
âcache-dir="/media/upload/vfscache"
âcache-info-age=28h
âcache-read-retries=3
âcache-chunk-total-size=10G
âcache-tmp-upload-path="/media/upload/pending"
âcache-tmp-wait-time=â2hâ
âcache-workers=5
âcache-writes
âconfig="/home/rclone/.config/rclone/rclone.conf"
âcontimeout=5s
âdir-cache-time=24h
âgid 0
âmax-read-ahead=128k
âtransfers=3
âtpslimit=3
âtimeout=5s
âtrack-renames
âuid 0
âumask 000
âvfs-cache-mode writes
-vv
When I tried running rsync, I got slightly better speeds, between 3.5 and 5 MB/s without playing any media. Still slow but better?