Too many request

I use rclone with this version:

rclone --version
rclone v1.37-088-gd54fca4e

  • os/arch: windows/amd64
  • go version: go1.8

When i run the command rclone check with dropbox this happend:

2017/08/31 11:39:10 Failed to check: error listing: Dropbox root ‘H_BKP’: too_ma
ny_requests/…

Thanks
Luis

Some of those messages are normal - did it carry on working afterwards?

No, after the error the program stop.

C:\Soft>rclone check C:\Users\Administrator.WINDOWS-MFUI7QK\Dropbox
\H_BKP Dpbx:H_BKP --exclude Thumbs.db --exclude .tmp --exclude ~.* --exclude D
esktop.ini --exclude .DS_Store --exclude desktop.ini -v --fast-list
2017/09/01 08:46:30 INFO : Dropbox root ‘H_BKP’: Modify window is 1s
2017/09/01 08:46:30 INFO : Local file system at \?\C:\Users\Administrator.WIND
OWS-MFUI7QK\Dropbox\H_BKP: Building file list
2017/09/01 08:46:30 INFO : Dropbox root ‘H_BKP’: Building file list
2017/09/01 08:51:56 ERROR : I+D/1. ALTA DE PRODUCTO/7. OPTIMIZACION DE PEDIDO A
PROVEEDOR/MULTITECH/TUEL/DIY-1691(GZ10)WHITE/3 UNIT BOX: error listing:
2017/09/01 08:51:56 ERROR : Dropbox root ‘H_BKP’: Error building file list: erro
r listing: Dropbox root ‘H_BKP’:
2017/09/01 08:51:57 Failed to check: error listing: Dropbox root ‘H_BKP’:

C:\Soft>

Can you send/attch/or paste me a complete log file with -vv --log-file rclone.log please?

Yes , I leave the log here:

https://drive.google.com/open?id=0B2bWxNNuPmAVaU45RWpKOU5VanM

Thankyou

Here I have another example, in another Pc, with a different dropbox client

rclone check “e:\BKPSERVERHP” dpbx:Bkps\Neologistic\BKPSERVERHP --exclude Thumbs.db --exclude .tmp --exclude ~.* --exclude Desktop.ini --exclude .DS_Store --exclude desktop.ini

2017/09/02 15:56:09 ERROR : compartida/color/InstallerContent/Help/assets/xml/svww: error listing: too_many_requests/…
2017/09/02 15:56:11 ERROR : compartida/Aguas/LICENCIAS NO AUTOMATICAS/Op 16016: error listing: too_many_requests/…
2017/09/02 15:56:21 ERROR : compartida/Carrefour y Norte/Textil/DJCP NUEVAS/Fotos DJCP/Viejas/PEM-AMERICA: error listing: too_many_requests/…
2017/09/02 15:56:23 ERROR : compartida/color/Installer/InstanceUpdate/Languages/zh-mo: error listing: too_many_requests/
2017/09/02 15:56:29 ERROR : compartida/Carrefour y Norte/LNA SIMI/9960263: error listing: too_many_requests/.
2017/09/02 15:56:31 ERROR : compartida/color/Installer/WSOptIn/Languages/tr: error listing: too_many_requests/…
2017/09/02 15:56:33 ERROR : compartida/PRUNE/LNA/OP.919: error listing: too_many_requests/…
2017/09/02 15:56:41 ERROR : compartida/b3net/Insixd/sim_out2: error listing: list continue: too_many_requests/…
2017/09/02 15:56:55 ERROR : compartida/Reld/LICENCIAS NO AUTOMATICAS/Op E-835: error listing: too_many_requests/.
2017/09/02 15:56:55 ERROR : Dropbox root ‘Bkps/Neologistic/BKPSERVERHP’: Error building file list: error listing: Dropbox root ‘Bkps/Neologistic/BKPSERVERHP’: too_many_requests/…
2017/09/02 15:56:55 Failed to check: error listing: Dropbox root ‘Bkps/Neologistic/BKPSERVERHP’: too_many_requests/…

is it possible to find a solution to this problem?
The error continues

Thanks

Thank you for the log. Eveything seems to be going very well for the first 5 minutes or so of the sync, then dropbox suddenly returns “error too_many_requests” for all your listing operations and none of the retries work.

Here is something similar on the dropbox forums.

They suggest multiple listings may be the problem…

So can you try setting --checkers 1 to see if that helps?

Can you also experiment with --tpslimit. The default will be 100, so try --tpslimit 10 and --tpslimit 1 and see if that makes a difference - I can change the default for dropbox if it does.

With the --checkers 1 the output was:

jos Técnicos)/DT colgantes/colg industria: error listing: too_many_requests/
2017/08/30 13:37:08 ERROR : Dropbox root ‘H_2daCopia’: Error building file list:
error listing: Dropbox root ‘H_2daCopia’: too_many_requests/.
2017/08/30 13:37:08 Failed to check: error listing: Dropbox root ‘H_2daCopia’: t
oo_many_requests/.

C:\Soft\BKP ROBOCOPY>rclone check C:\Users\Administrator.IDEA\Dropbox\H_2daCopia
Dpbx:H_2daCopia --exclude Thumbs.db --exclude .tmp --exclude ~.* --exclude De
sktop.ini --exclude .DS_Store --exclude desktop.ini --checkers 1

C:\Soft\BKP ROBOCOPY>rclone check C:\Users\Administrator.IDEA\Dropbox\H_2daCopia
Dpbx:H_2daCopia --exclude Thumbs.db --exclude .tmp --exclude ~.* --exclude De
sktop.ini --exclude .DS_Store --exclude desktop.ini --checkers 1 -v
2017/09/06 08:25:57 INFO : Dropbox root ‘H_2daCopia’: Modify window is 1s
2017/09/06 08:25:57 INFO : Local file system at \?\C:\Users\Administrator.IDEA
\Dropbox\H_2daCopia: Building file list
2017/09/06 08:25:57 INFO : Dropbox root ‘H_2daCopia’: Building file list
2017/09/06 10:41:16 ERROR : I+D/14. VARIOS/INGENIERIA/[ I + D ]/WEB/WEB 2012/Fic
has técnicas/10 exterior/Guio-Mini Guio: error listing:
2017/09/06 10:41:17 ERROR : Dropbox root ‘H_2daCopia’: Error building file list:
error listing: Dropbox root ‘H_2daCopia’:
2017/09/06 10:41:17 Failed to check: error listing: Dropbox root ‘H_2daCopia’:

Now I will try with --tpslimit

A new error appears JSON input

C:\Soft\BKP ROBOCOPY>rclone check C:\Users\Administrator.IDEA\Dropbox\H_2daCopi
Dpbx:H_2daCopia --exclude Thumbs.db --exclude .tmp --exclude ~.* --exclude D
sktop.ini --exclude .DS_Store --exclude desktop.ini --checkers 1 --tpslimit 1 -

2017/09/06 11:09:33 INFO : Starting HTTP transaction limiter: max 1 transactio
s/s with burst 1
2017/09/06 11:09:34 INFO : Dropbox root ‘H_2daCopia’: Modify window is 1s
2017/09/06 11:09:34 INFO : Local file system at \?\C:\Users\Administrator.IDE
\Dropbox\H_2daCopia: Building file list
2017/09/06 11:09:34 INFO : Dropbox root ‘H_2daCopia’: Building file list
2017/09/06 12:05:56 ERROR : I+D/6. PROCESOS Y PROCEDIMIENTOS/1. AUDITORIA/5 -CA
IBRACION DE EQUIPOS DE ENSAYO Y MEDICION (PE03 - FR05)/HISTORICO: error listing
unexpected end of JSON input
2017/09/06 12:05:57 ERROR : Dropbox root ‘H_2daCopia’: Error building file list
error listing: Dropbox root ‘H_2daCopia’: unexpected end of JSON input
2017/09/06 12:05:57 Failed to check: error listing: Dropbox root ‘H_2daCopia’:
nexpected end of JSON input

C:\Soft\BKP ROBOCOPY>

:frowning: That is likely the server giving a 500 error with no error message.

How many files have you got on your dropbox? I’d like to see if I can replicate this.

I ran the command rclone size dpbx:/ -q and it throw many errors and the total size.

2017/09/07 12:47:06 ERROR : Estudio Pino BKP/f/udid/653601F0657.c141cf98dd31bc58
faa2d52b58ae1c6f.b64: Entry doesn’t belong in directory "Estudio Pino BKP/F/udid
" (too short) - ignoring
Total objects: 56770
Total size: 137.165 GBytes (147279799090 Bytes)

thanks for taking care

That issue is fixed in the latest beta - see #1558

This is the output

C:\Soft\rclone>rclone size DpbxT:
2017/09/08 11:22:50 ERROR : Bkps/2daCOPIA/Smurra/documentos/Soft/PENTACAM: error listing:
2017/09/08 11:22:50 Failed to size:

C:\Soft\rclone>rclone size DpbxT:
2017/09/08 11:27:53 ERROR : Bkps/Borre/cartas/AFI BROKERS INMOBILIARIA/ROMINA/CIA/Cia Consultor/Historia de Arte II/fotos para el final/Paleocristiana: error listing:
2017/09/08 11:27:53 Failed to size:

C:\Soft\rclone>rclone --version
rclone v1.37-088-gd54fca4e

  • os/arch: windows/amd64
  • go version: go1.8

if I uploaded many small files to dropbox, I’m having the same problem with “too many requests” is it possible to stop uploading the rclone as soon as the message appears like a break command as switch?

2017/10/04 03:34:39 ERROR : Uel7nnCHvS-6sWiCYh-vBqwl: Dir.Lookup error: too_many_requests/
2017/10/04 03:34:59 ERROR : M6utavRZNqOxc0: Dir.Lookup error: too_many_requests/…
2017/10/04 03:34:19 ERROR : Uel7nnCHvS-6sWiCYh-vBqwl: Dir.Lookup error: too_many_requests/…
2017/10/04 03:34:39 INFO :
Transferred: 1536.232 GBytes (36.414 MBytes/s)

the reason why I’m asking, when the message “too many requests” comes hangs the rclone mount and the load of my server goes upstairs, when I then kill rclone with the upload, it takes about 2 minutes for the rclone mount to run normally again, as if the mount would freeze in the time when there are too many requests.

I dont know, I cant use rclone anymore because too many request is always present.

have you tried the current version v1.38?

Try the --tpslimit flag - that might help with limiting the files per second which is what dropbox is complaining about.

rclone does exponential backoff on errors which is the correct thing to do according to dropbox I think.

I download the new version and is working properly again!

thanks PatriotM!!

1 Like