Dropbox mount rate limiting

What is the problem you are having with rclone?

Current issue is with dropbox remote I always keep getting the rate limit issue with various different configurations.

2023/06/30 23:29:45 NOTICE: too_many_requests/.: Too many requests or write operations. Trying again in 300 seconds.

I have tested with tpslimit 12 tpsburst 12, tpslimit 12 tpsburt 0, tpslimit 5 tpsburst 0 and now recently with 1.63 `tpslimit 5 tpsburst 0 pacer 100ms'

What happens is usually the mount folder becomes unresponsive and I am unable to ls the directory.

Now what I don't understand is when I run rclone lsd remote: I am still able to list directories. I tried catting a file with rclone and it does work.

If I decide to restart rclone then it does work so I am unsure why the limit triggers but I am still able to continue with using the commands.

I removed caching just to make sure everything runs fine without cache. I can implement cache later.

Now I do have two mounts one for plex and the rest for services like sonarr, radarr, etc. On the plex mount this rate limiting issue is never hit and I don't even have a tpslimit there. Now the difference between them is I would think these services are heavier on the scanning side. Now I don't mind it being slow as long as I can stay within t he api limit but it seems like whatever I try I still hit the same issues.

So now I am wondering are these flags not being respected for all api requests that rclone is sending or do I have something wrong with the way I am using rclone?

To note these mounts are read only and no upload happens there. And each mount has its seperate app so client_id and secret is not the same

Run the command 'rclone version' and share the full output of the command.

rclone 1.63
also tested on 1.60 and 1.59 but without the new pacer flag

Which cloud storage system are you using? (eg Google Drive)

Dropbox

The command you were trying to run (eg rclone copy /tmp remote:tmp)

mount --config=/config/rclone/rclone.conf --dropbox-batch-mode off --fast-list --dropbox-pacer-min-sleep 100ms  --tpslimit 5 --tpslimit-burst 0  --allow-non-empty --log-level=ERROR --log-level=INFO --log-level=NOTICE  --log-file=/config/rclone/rclone.log --allow-other cryptd1: ${DROPBOX}

The rclone config contents with secrets removed.

[dropbox]
type = dropbox
token = 
client_id = 
client_secret = 

[cryptd1]
type = crypt
remote = dropbox:/PMS
password = 
password2 = 

A log from the command with the -vv flag

2023/06/30 22:37:01 NOTICE: --fast-list does nothing on a mount
2023/06/30 23:01:46 NOTICE: too_many_requests/.: Too many requests or write operations. Trying again in 300 seconds.
2023/06/30 23:11:23 NOTICE: too_many_requests/...: Too many requests or write operations. Trying again in 300 seconds.
2023/06/30 23:29:42 NOTICE: too_many_requests/..: Too many requests or write operations. Trying again in 300 seconds.
2023/06/30 23:29:43 NOTICE: too_many_requests/.: Too many requests or write operations. Trying again in 300 seconds.
2023/06/30 23:29:44 NOTICE: too_many_requests/: Too many requests or write operations. Trying again in 300 seconds.
2023/06/30 23:29:44 NOTICE: too_many_requests/.: Too many requests or write operations. Trying again in 300 seconds.
2023/06/30 23:29:45 NOTICE: too_many_requests/.: Too many requests or write operations. Trying again in 300 seconds.
2023/06/30 23:54:41 NOTICE: too_many_requests/: Too many requests or write operations. Trying again in 300 seconds.

change log level --log-level=DEBUG and post log

Before testing you can try to recreate dropbox remote configuration (delete and create again) as it often happens that token is not bound to personal client_id

Have a look at this post:

Dropbox does it rate limit per app configured.

if you have more than one thing using the app id, it'll count to the rate limiting.

Best to create multiple app registrations for everything.

I have them as separate apps

Can't see what's on your computer. Only what you've got posted.

You've got a single mount command and a single key in your rclone.conf in your OP.

If you are running more, the more information you share, the easier and less time wasted getting to an answer.

In the dropbox activity logs you can see which apps are used and I confirmed that it is using the apps that I created vs the regular rclone app.

The logs for a couple of minutes where huge like about 100k and uploading it to pastebin was just causing my browser to freeze. So I took stuff before the too_many_requests and after it. And the head of the file is the block below.

2023/07/02 06:18:38 INFO  : Starting transaction limiter: max 5 transactions/s with burst 1
2023/07/02 06:18:38 DEBUG : rclone: Version "v1.63.0" starting with parameters ["rclone" "mount" "--config=/config/rclone/rclone.conf" "--dropbox-batch-mode" "off" "--fast-list" "--dropbox-pacer-min-sleep" "200ms" "--tpslimit" "5" "--tpslimit-burst" "0" "--allow-non-empty" "-vv" "--log-file=/config/rclone/rclone.log" "--allow-other" "cryptd1:" "/mnt/storage/dropbox-rclone"]
2023/07/02 06:18:38 NOTICE: --fast-list does nothing on a mount
2023/07/02 06:18:38 DEBUG : Creating backend with remote "cryptd1:"
2023/07/02 06:18:38 DEBUG : Using config file from "/config/rclone/rclone.conf"
2023/07/02 06:18:39 DEBUG : Creating backend with remote "dropbox:/PMS"
2023/07/02 06:18:39 DEBUG : dropbox: detected overridden config - adding "{5otOt}" suffix to name
2023/07/02 06:18:39 DEBUG : Dropbox root '': Using root namespace "4108663377"
2023/07/02 06:18:39 DEBUG : fs cache: renaming cache item "dropbox:/PMS" to be canonical "dropbox{5otOt}:PMS"
2023/07/02 06:18:39 DEBUG : Encrypted drive 'cryptd1:': Mounting on "/mnt/storage/dropbox-rclone"
2023/07/02 06:18:39 DEBUG : : Root: 
2023/07/02 06:18:39 DEBUG : : >Root: node=/, err=<nil>
2023/07/02 06:18:43 DEBUG : /: Attr: 
2023/07/02 06:18:43 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2023/07/02 06:18:56 DEBUG : /: Lookup: name="Music"
2023/07/02 06:18:56 DEBUG : /: >Lookup: node=Music/, err=<nil>
2023/07/02 06:18:56 DEBUG : Music/: Attr: 

I did mention that I used seperate apps here.

My rclone conf has more stuff because of older remotes and stuff and I wanted to simplify things here so its easier to know what the issue is. In my experience if I just dump conf and walls of text it will end up not being read.

So for more information:
I have 2 mounts and an upload remote. All 3 of them use different apps with different app id and secrets and they use different users on dropbox to separate them.

I have no issues on the two other remotes the remote in question is the one I posted above that has the issue.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.