Dropbox Dir.ReadDirAll error: too_many_requests/

This is already answered with a solution and a root cause.

If the solution doesn't work for you, start a new post and use the help / support template and fill it out completely.

Hi @plotterx and @Hunny,

The below post seems relevant in your situation. It was written about OneDrive but the same behaviour and logic applies for Dropbox:

This may also interest you:

I've done quite the load testing/validation on the TPS limits for Dropbox and engaged support (who doesn't quite answer much unfortunately).

The general consensus is 12 tps per app registration:

dropbox: uploading improvements · Issue #5156 · rclone/rclone (github.com)

It's very easy to test/generate the error as rclone itself is really fast and hits it very fast if you have a mount with many directories. There is a sweet spot you hit when have too many directories on a mount as I didn't hit it at first either.

@Animosity022 Seems like you missed my point. I am not questioning your observations and knowledge in this area - on the contrary.

I am trying to explain why services like Dropbox, OneDrive, Google Drive etc. all have this kind of limitations; and give some food for thought on Chia farming in general.

Hello,

Your point is so meaningful but I hope that's not the reason. I've already tried over an app without direct mount over rclone. But I got the same errors..

Are there any other solutions from you guys?

Also it gets the same error even with tps limit 1...

rclone mount dropbox4:/Party1 /dropbox4 --tpslimit 1 --allow-non-empty --log-level DEBUG
2022/02/07 16:48:25 INFO : Starting transaction limiter: max 1 transactions/s with burst 1
2022/02/07 16:48:25 DEBUG : rclone: Version "v1.58.0-beta.5989.aa2d7f00c" starting with parameters ["rclone" "mount" "dropbox4:/Party1" "/dropbox4" "--tpslimit" "1" "--allow-non-empty" "--log-level" "DEBUG"]
2022/02/07 16:48:25 DEBUG : Creating backend with remote "dropbox4:/Party1"
2022/02/07 16:48:25 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2022/02/07 16:48:26 DEBUG : Dropbox root '': Using root namespace "2201984529"
2022/02/07 16:48:27 DEBUG : fs cache: renaming cache item "dropbox4:/Party1" to be canonical "dropbox4:Party1"
2022/02/07 16:48:28 DEBUG : Dropbox root 'Party1': Mounting on "/dropbox4"
2022/02/07 16:48:28 DEBUG : : Root:
2022/02/07 16:48:28 DEBUG : : >Root: node=/, err=
2022/02/07 16:48:30 DEBUG : /: Lookup: name="plot1"
2022/02/07 16:48:31 DEBUG : /: >Lookup: node=plot1/, err=
2022/02/07 16:48:31 DEBUG : plot1/: Attr:
2022/02/07 16:48:31 DEBUG : plot1/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2022/02/07 16:48:31 DEBUG : plot1/: ReadDirAll:
2022/02/07 16:48:34 NOTICE: too_many_requests/..: Too many requests or write operations. Trying again in 5 seconds.
2022/02/07 16:48:34 DEBUG : pacer: low level retry 1/10 (error too_many_requests/..)
2022/02/07 16:48:34 DEBUG : pacer: Rate limited, increasing sleep to 5s
2022/02/07 16:48:36 NOTICE: too_many_requests/.: Too many requests or write operations. Trying again in 5 seconds.
2022/02/07 16:48:36 DEBUG : pacer: low level retry 2/10 (error too_many_requests/.)
2022/02/07 16:48:41 NOTICE: too_many_requests/: Too many requests or write operations. Trying again in 5 seconds.
2022/02/07 16:48:41 DEBUG : pacer: low level retry 3/10 (error too_many_requests/)
2022/02/07 16:48:46 NOTICE: too_many_requests/..: Too many requests or write operations. Trying again in 5 seconds.
2022/02/07 16:48:46 DEBUG : pacer: low level retry 4/10 (error too_many_requests/..)

I know that once OneDrive starts the hard rate limiting (too many requests), then it takes some time until you are allowed a normal rate again - up to 24 hours. The less activity you have in this period, the shorter. I guess you will see something similar on Dropbox.

Sorry, others have been there before:

My comment was meant to be additive like yours :slight_smile:

We have the help template for filling in things and getting information.

Did you register your own app? Are you using one app for the mount like I suggested? Can you see what the API usage is for that app?

What's the output of your rclone version?
What's your rclone.conf look like?

1 Like

Did you register your own app?

Yes

Are you using one app for the mount like I suggested?

Yes I've registered an app to use that.

Can you see what the API usage is for that app?

I don't know how to check this.

What's the output of your rclone version?

rclone v1.58.0-beta.5989.aa2d7f00c
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.4.0-97-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.17.6
- go/linking: static
- go/tags: none

Developers - Dropbox - Click on App Console. Click the App you registered, click on the analytics.

If it's working, you'll see data:

If that's the only app registered to that, you should see 1 user registered against it and the API usage. If you are generate an error with 1 TPS, something is very wrong.

Post a full debug log with that mount.

Why are you using a beta version as well? If there isn't a specific reason, please use the stable.

The reason to use beta version is, try to test if you have fixed the problem on beta version or not.

Okay I've installed the latest version.

rclone v1.57.0
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.4.0-97-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.17.2
- go/linking: static
- go/tags: none

When I connect the app and just with ls command, I can see this logs:

fusermount -uz /dropbox4
rclone mount dropbox4:/Party1 /dropbox4 --log-level DEBUG --allow-non-empty --tpslimit 1
fusermount: failed to unmount /dropbox4: Invalid argument
fusermount: failed to unmount /dropbox4: Invalid argument
2022/02/07 19:14:59 INFO : Starting transaction limiter: max 1 transactions/s with burst 1
2022/02/07 19:14:59 DEBUG : rclone: Version "v1.57.0" starting with parameters ["rclone" "mount" "dropbox4:/Party1" "/dropbox4" "--log-level" "DEBUG" "--allow-non-empty" "--tpslimit" "1"]
2022/02/07 19:14:59 DEBUG : Creating backend with remote "dropbox4:/Party1"
2022/02/07 19:14:59 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2022/02/07 19:15:00 DEBUG : Dropbox root '': Using root namespace "2201984529"
2022/02/07 19:15:01 DEBUG : fs cache: renaming cache item "dropbox4:/Party1" to be canonical "dropbox4:Party1"
2022/02/07 19:15:02 DEBUG : Dropbox root 'Party1': Mounting on "/dropbox4"
2022/02/07 19:15:02 DEBUG : : Root:
2022/02/07 19:15:02 DEBUG : : >Root: node=/, err=
2022/02/07 19:15:07 DEBUG : /: Lookup: name="plot1"
2022/02/07 19:15:07 DEBUG : /: >Lookup: node=plot1/, err=
2022/02/07 19:15:07 DEBUG : plot1/: Attr:
2022/02/07 19:15:07 DEBUG : plot1/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2022/02/07 19:15:07 DEBUG : plot1/: ReadDirAll:
2022/02/07 19:15:09 NOTICE: too_many_requests/.: Too many requests or write operations. Trying again in 5 seconds.
2022/02/07 19:15:09 DEBUG : pacer: low level retry 1/10 (error too_many_requests/.)
2022/02/07 19:15:09 DEBUG : pacer: Rate limited, increasing sleep to 5s
2022/02/07 19:15:11 NOTICE: too_many_requests/: Too many requests or write operations. Trying again in 5 seconds.
2022/02/07 19:15:11 DEBUG : pacer: low level retry 2/10 (error too_many_requests/)
2022/02/07 19:15:15 NOTICE: too_many_requests/: Too many requests or write operations. Trying again in 5 seconds.
2022/02/07 19:15:15 DEBUG : pacer: low level retry 3/10 (error too_many_requests/)
2022/02/07 19:15:20 NOTICE: too_many_requests/..: Too many requests or write operations. Trying again in 5 seconds.
2022/02/07 19:15:20 DEBUG : pacer: low level retry 4/10 (error too_many_requests/..)
2022/02/07 19:15:25 NOTICE: too_many_requests/: Too many requests or write operations. Trying again in 5 seconds.
2022/02/07 19:15:25 DEBUG : pacer: low level retry 5/10 (error too_many_requests/)
2022/02/07 19:15:30 NOTICE: too_many_requests/...: Too many requests or write operations. Trying again in 5 seconds.
2022/02/07 19:15:30 DEBUG : pacer: low level retry 6/10 (error too_many_requests/...)
2022/02/07 19:15:35 NOTICE: too_many_requests/..: Too many requests or write operations. Trying again in 5 seconds.
2022/02/07 19:15:35 DEBUG : pacer: low level retry 7/10 (error too_many_requests/..)
2022/02/07 19:15:40 NOTICE: too_many_requests/..: Too many requests or write operations. Trying again in 5 seconds.
2022/02/07 19:15:40 DEBUG : pacer: low level retry 8/10 (error too_many_requests/..)
2022/02/07 19:15:45 NOTICE: too_many_requests/...: Too many requests or write operations. Trying again in 5 seconds.
2022/02/07 19:15:45 DEBUG : pacer: low level retry 9/10 (error too_many_requests/...)
2022/02/07 19:15:50 NOTICE: too_many_requests/.: Too many requests or write operations. Trying again in 5 seconds.
2022/02/07 19:15:50 DEBUG : pacer: low level retry 10/10 (error too_many_requests/.)
2022/02/07 19:15:50 DEBUG : plot1: Dir.ReadDirAll error: too_many_requests/.
2022/02/07 19:15:50 DEBUG : plot1/: >ReadDirAll: item=-1, err=too_many_requests/.
2022/02/07 19:16:02 DEBUG : Dropbox root 'Party1': Checking for changes on remote

I've created a new app to test your orders but I can't see the new app's statistics in 1 day.

This is the old app's statistics.


r

The old app was getting hammered though.

10 users. Your per day is quite high as it seems to make sense that app was over loaded.

If you do 12 TPS continually for 24 hours, you get 10.3m requests in a day.

To the best of my knowledge, you can make as many apps as you want so I use 4 total.

I have 2 mounts that each use 1.
I have 1 upload app
I have 1 play around app.

Each is limited to 12 tps and since I've done that, I've never had a TPS issue.

I'll wait until tomorrow, And I'll post the newest results. The machine is power off now. We will see it

hello,
I've created a new app and deleted the old one. And connected with only and just with a simple connection parameters such:

rclone mount dropbox4:/Party1 /dropbox4 --log-level DEBUG --allow-non-empty --tpslimit 1

Also I'm trying to get help from dropbox but they told me and all of my friends this is not about dropbox. There is no api update. So please can you contact with dropbox to solve this? Because they request the developers to contact them.

As I told before, I can provide rclone config for debugging or etc.

Anyway, but unfortunately the same error is continued...

2022/02/08 18:57:06 INFO  : Starting transaction limiter: max 1 transactions/s with burst 1
2022/02/08 18:57:06 DEBUG : rclone: Version "v1.57.0" starting with parameters ["rclone" "mount" "dropbox4:/Party1" "/dropbox4" "--log-level" "DEBUG" "--allow-non-empty" "--tpslimit" "1"]
2022/02/08 18:57:06 DEBUG : Creating backend with remote "dropbox4:/Party1"
2022/02/08 18:57:06 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2022/02/08 18:57:07 DEBUG : Dropbox root '': Using root namespace "2201984529"
2022/02/08 18:57:08 DEBUG : fs cache: renaming cache item "dropbox4:/Party1" to be canonical "dropbox4:Party1"
2022/02/08 18:57:09 DEBUG : Dropbox root 'Party1': Mounting on "/dropbox4"
2022/02/08 18:57:09 DEBUG : : Root:
2022/02/08 18:57:09 DEBUG : : >Root: node=/, err=<nil>
2022/02/08 18:57:25 DEBUG : /: Attr: 
2022/02/08 18:57:25 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2022/02/08 18:57:25 DEBUG : /: ReadDirAll: 
2022/02/08 18:57:25 DEBUG : /: >ReadDirAll: item=5, err=<nil>
2022/02/08 18:57:25 DEBUG : /: Lookup: name="plot1"
2022/02/08 18:57:25 DEBUG : /: >Lookup: node=plot1/, err=<nil>
2022/02/08 18:57:25 DEBUG : plot1/: Attr: 
2022/02/08 18:57:25 DEBUG : plot1/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2022/02/08 18:57:25 DEBUG : /: Lookup: name="plot2"
2022/02/08 18:57:25 DEBUG : /: >Lookup: node=plot2/, err=<nil>
2022/02/08 18:57:25 DEBUG : plot2/: Attr: 
2022/02/08 18:57:25 DEBUG : plot2/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2022/02/08 18:57:25 DEBUG : /: Lookup: name="plot3"
2022/02/08 18:57:25 DEBUG : /: >Lookup: node=plot3/, err=<nil>
2022/02/08 18:57:25 DEBUG : plot3/: Attr: 
2022/02/08 18:57:25 DEBUG : plot3/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2022/02/08 18:57:34 DEBUG : /: Attr: 
2022/02/08 18:57:34 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2022/02/08 18:57:42 DEBUG : /: Lookup: name="plot1"
2022/02/08 18:57:42 DEBUG : /: >Lookup: node=plot1/, err=<nil>
2022/02/08 18:57:42 DEBUG : plot1/: Attr: 
2022/02/08 18:57:42 DEBUG : plot1/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2022/02/08 18:57:42 DEBUG : plot1/: ReadDirAll: 
2022/02/08 18:57:59 NOTICE: too_many_requests/..: Too many requests or write operations. Trying again in 5 seconds.
2022/02/08 18:57:59 DEBUG : pacer: low level retry 1/10 (error too_many_requests/..)
2022/02/08 18:57:59 DEBUG : pacer: Rate limited, increasing sleep to 5s

btw, I've a guess, I can see a folder which has several items. Maybe there is a limitation about total file folder size or item count. If api have ability, you can read page by page the directory except reading all.

You are only getting 5 items in a directory and it lists them all out.

and then you try to relist that same directory again and it errors out.

Dropbox won't tell you, me or @ncw about what rate limit you have crossed as they won't share that information and they'll refer you back to the respecting the rate limits and it's not documented or shared.

You'll get a generic response like:

So they won't offer anymore information.

The part that is odd for me is the 5 second back off as I've never seen that and I've only see 300 seconds in any my logs.

I'm not aware of any hard limit but you have 5 items so that doesn't seem like you are crossing it. I have one directory with 5k folders in it.

What does the API graph show for the one you just created? Are you seeing any odd spikes there? tpslimit 1 should reduce you to 1 transaction per second and if that's tripping their rate limits on your App, something is very wrong with that.

Yeah, I guess I'm right.
Because when I move some files about 2 TB to another directory, I can list the new directory. So it can be about max file size or max file count. I'm a developer too. But I'm not experienced on golang. I've implemented a similar solution about fething more data than the system lets. As I said, I can list a folder which have less files in it. The only way to pass this, listing files page by page such as 100 with 9 times for 900 files

Sorry as I'm not following what you are doing. Can you be more specific with steps?

In the last scenario I shared, I got the same errors as you know. But I suspect this problem is related to number of files or total size of the directory. Then I moved around 20 files to different folder and tried to list that folder. Successfully listed. So I think about a solution but I have no experience on it.
What I request you is, implementing a turnaround solution.
If you can implement the readalldir function as page by page, the problem can be solved

Without knowing what you are doing and not being able to peer into your computer this is very tough to help you out.

Based on what? What is the number of files? What's the size of the directory?

What was the folder count before? What is the size?

You haven't even explained what the issue is and you are asking to code a solution?

  • I had 900 files before this operation. (in Party1/plot2)
  • Each file is 100 GB, total folder size was 90 TB. (Party1/plot2)
  • In the final, source folder has 872 files (87.2 TB) (Party1/plot2) and the target folder is 28 files (2.8 TB).(Party1/plot3)

each folder is in the same hierarchic level.

Totally, I've 3 main folder names are Party,
Each folder have 2 folders names are plot1 and plot2
Each plot folders have 900 files each 100 GB