Recommended Dropbox (Formally Google Drive) and Plex Mount Settings

For me, anymore than 12 TPS generates rate limit errors so I don't go higher than that.

That's not quite how ffprobe/mediainfo work as it jumps around a bit. Plex opens/closes a file 3 times when it analyzes and that generally happens in the background when something is scanned. Generally, if you are scanning a brand new library with TBs of data, you'd see some difference as you want to reduce the waste.

Scans aren't relevant as once it's scanned, it's just dir cache time as those are instant as it's only checking file size/mod time on the files. Playback with a tiny chunk size actually takes longer as it has to make many more API hits / range requests to get the data as it's reading sequentially to start playback (assuming the file is analyzed already).

To solve what? You can't easily manage that with Sonarr/Radarr. Once the library is analyzed the first time, everything else is pretty much instant. I don't break things up to avoid having to manage my stuff more as I really just want to set and forget and if it takes me 1.5 seconds to start vs 1 second, that's alright in my book as I do nothing to manage it.

That's why I removed mergerfs from setup as I have situations where I have double the data for a period of time until it's upload and I can't use hardlinks but I have one less piece of software and I remove an upload script.

In the end, whatever works best for you, use that. I use what works best for me and my setup as my goals/use case might be different and just as "correct" as yours is for you. Isn't really a 'right' in this case as what works best for you, do that.

That's right. I probably going to test some stuff and at the moment it works quite well.

Hmm. Where are you getting the error?
I never received one error from Dropbox because Rate Limit or similar.
I tested as mounted Volume for Plex with a new fresh library and scan, with chapter thumbnail generation, thumbnail preview generation, intro detection.

I did a copy with 10000 files (1MB each) at transfers 64.
I did a simple ncdu and lsl with TPS limit 100 for the sake of completion at three different servers and on my local machine with the same client id (app).

I only noticed rclone is demanding many resources to manage the many requests but that's normal if you throw everything at it.

EDIT:
I noticed something. If I add one server more with tpslimit 100 I recieve the error of "too many requests". Makes sense since 5 devices with high bandwidth and fast latency are accessing the poor app all at once.

Could be something has changed. Many folks tested dropbox and here are a number of results here:

dropbox: uploading improvements · Issue #5156 · rclone/rclone (github.com)

Are you setting just tpslimit or changing burst as well?

This breaks super fast.

rclone size DB: -vv --tpslimit 100 --tpslimit-burst 100
2022/10/17 12:24:35 DEBUG : Setting --config "/opt/rclone/rclone.conf" from environment variable RCLONE_CONFIG="/opt/rclone/rclone.conf"
2022/10/17 12:24:35 INFO  : Starting transaction limiter: max 100 transactions/s with burst 100
2022/10/17 12:24:35 DEBUG : rclone: Version "v1.59.2" starting with parameters ["rclone" "size" "DB:" "-vv" "--tpslimit" "100" "--tpslimit-burst" "100"]
2022/10/17 12:24:35 DEBUG : Creating backend with remote "DB:"
2022/10/17 12:24:35 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2022/10/17 12:25:12 NOTICE: too_many_requests/.: Too many requests or write operations. Trying again in 300 seconds.
2022/10/17 12:25:12 DEBUG : pacer: low level retry 1/10 (error too_many_requests/.)
2022/10/17 12:25:12 DEBUG : pacer: Rate limited, increasing sleep to 5m0s
2022/10/17 12:25:12 NOTICE: too_many_requests/...: Too many requests or write operations. Trying again in 300 seconds.
2022/10/17 12:25:12 DEBUG : pacer: low level retry 1/10 (error too_many_requests/...)
2022/10/17 12:25:12 DEBUG : pacer: Reducing sleep to 3m45s
2022/10/17 12:25:12 DEBUG : pacer: Reducing sleep to 2m48.75s
2022/10/17 12:25:12 DEBUG : pacer: Reducing sleep to 2m6.5625s
2022/10/17 12:25:12 DEBUG : pacer: Reducing sleep to 1m34.921875s
2022/10/17 12:25:12 DEBUG : pacer: Reducing sleep to 1m11.19140625s
2022/10/17 12:25:12 DEBUG : pacer: Reducing sleep to 53.393554687s
2022/10/17 12:25:12 DEBUG : pacer: Reducing sleep to 40.045166015s
https://github.com/rclone/rclone/issues/5156^C2022/10/17 12:25:22 INFO  : Signal received: interrupt
2022/10/17 12:25:22 INFO  : Dropbox root '': Commiting uploads - please wait...
2022/10/17 12:25:22 INFO  : Exiting...

That is very interesting.
My result at 100.

PS C:\Users\Nikolas Thiel> rclone size Dropbox-test: -vv --tpslimit 100 --tpslimit-burst 100
2022/10/17 18:36:23 INFO  : Starting transaction limiter: max 100 transactions/s with burst 100
2022/10/17 18:36:23 DEBUG : rclone: Version "v1.59.1" starting with parameters ["C:\\PATH\\rclone.exe" "size" "Dropbox-test:" "-vv" "--tpslimit" "100" "--tpslimit-burst" "100"]
2022/10/17 18:36:23 DEBUG : Creating backend with remote "Dropbox-test:"
2022/10/17 18:36:23 DEBUG : Using config file from "rclone.conf"
Total objects: 30.266k (30266)
Total size: 9.888 TiB (10872462775685 Byte)
2022/10/17 18:37:15 DEBUG : 6 go routines active
2022/10/17 18:37:15 INFO  : Dropbox root '': Commiting uploads - please wait...

This is on my test dropbox folder.

Strange?

Does seem very odd. I can pretty much break anything past around 12-13 tps per second.

You are using your own API key and are you testing or production? I'm not aware of any other settings that would impact it as your results seem very different from everyone else.

image

This seems indeed strange. Well, i won't complain but i did a full run of my libary from my plex server.
With --tpslimit 500 and burst.

This time, i used somewhat the tps of 500. No errors but the run completed.

root@ubuntu-2gb-fsn1-1:~# rclone size Dropbox: -vv --tpslimit 500 --tpslimit-burst 500
2022/10/17 17:04:55 INFO  : Starting transaction limiter: max 500 transactions/s with burst 500
2022/10/17 17:04:55 DEBUG : rclone: Version "v1.59.2" starting with parameters ["rclone" "size" "Dropbox:" "-vv" "--tpslimit" "500" "--tpslimit-burst" "500"]
2022/10/17 17:04:55 DEBUG : Creating backend with remote "Dropbox:"
2022/10/17 17:04:55 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
Total objects: 756.650k (756.650)
Total size: 494.427 TiB (543628681584250 Byte)
2022/10/17 17:09:44 DEBUG : 6 go routines active
2022/10/17 17:09:44 INFO  : Dropbox root '': Commiting uploads - please wait...

My mount setting for my plex (i run plex and rclone with docker):

rclone mount Dropbox: /data --allow-other --dir-cache-time 9999h --log-file /logs/rclone.log --log-level INFO --umask 002 --cache-dir=/cache --vfs-cache-mode full --vfs-cache-max-size 800G --vfs-write-back 1h --vfs-cache-max-age 24h --tpslimit 32 --tpslimit-burst 32 --disable-http2 --attr-timeout 1s --vfs-read-chunk-size 10M --allow-non-empty

Did you make that recently or old? I really can't explain why that would be so different from one user to the next.

Well, i don't know if this helps, but my app got created 3 months ago and i set an image, description, puplisher and website link

Hmm - I'll just give it a whirl and see.

I'm interested at the result- :slight_smile:
I highly doubt it or something is going "wrong" with my buissnes account.

I wonder if there is some scale per user registered? I only have 1 on each of mine. New app performed just as bad and bombed out pretty quick.

@Animosity022 quick question about your setup if you don't mind

As you have a separate Dropbox app for your film and tv content, I presume that means you also need to have separate mounts, correct? Does that also mean you have more than one cache?

Up until now I've been running two mounts, one for my Radarr, Sonarr and any uploads and another for Plex. The idea was to use Bwlimit on the Radarr/Sonarr mount so that I can manage traffic during peak hours, without also throttling Plex

I'm also curious about your 1 hour upload delay. What is the purpose of that?

Yes.

Yes.

Generally, my thought process was to give things time to "settle" before I uploaded them as I have a 2TB drive that does my caching so I didn't want to fill it up. I've got FIOS so don't care about uploading and downloading at the same time so need to bwlimiting from my perspective.

I use QoS on my OPNSense router so traffic is shared pretty much evenly across sessions so no one session can overpower anything else. My setup is definitely very well tuned for my use case and I spent a lot of time on the QoS portion testing various setups and finally landed on this after some time as it just works well.

Unfortunately my router only has dynamic QoS, so I can't configure bandwidth that way.

Can you see a downside to me having a separate Dropbox app/mount for my uploads, as well as mounts for TV and Movies? (actually there would be 5 mounts as I separate 4k tv and 4k movies)

The only negative I can think of would be smaller caches. I have two 1TB SSD drives, so I could share the space between the 5 caches

Nope. I don't see any issues with that. I generally say, use what works best for your setup as my flow has been a constant optimization for years based on my use case.

@Animosity022 I've finally transfered all my content from Gdrive to Dropbox and I'm in the process of setting up my Mounts, however I'm running into some issues with the DropBox apps. I've set them up as per the rclone guide however I'm getting the following error when I set up my remotes

Fatal error: failed to get token: oauth2: cannot fetch token: 400 Bad Request
Response: {"error": "invalid_client: Invalid client_id or client_secret"}

Is this something you've run into?

Best to post a new thread for support questions as it's not related to this one.

I haven't use Google Drive for months at this point.

Sorry if I wasn't clear. I'm emulating your setup with the Dropbox mounts, but despite following the instructions i am getting issues with the client ID and client secret

Best to start a new help/support thread as your question is not with my settings but a general help question.

Fair enough, will do