Rclone Beta - New switch(s) ! woohoo :-)

Help and Support
1

Hi guys,

I know there is quota on Gsuite ect… and the error below is current.

Failed to copy: googleapi: Error 403: User Rate Limit Exceeded, userRateLimitExceeded

-BUT-

I wonder if you are using better switch(s) / parameters than i tested already.

In the last Beta

rclone v1.37-093-gb510c70cβ

  • os/arch: linux/amd64
  • go version: go1.8.3

rclone copyto /source/dir/ gsuite:/destination/dir --verbose --no-check-certificate --no-traverse --no-gzip-encoding --timeout=600s --drive-use-trash --drive-chunk-size=512M --drive-upload-cutoff=1G --buffer-size=1G --ignore-existing --no-update-modtime --low-level-retries=6 --retries=4 --stats=15m --transfers=32 --checkers=64 --log-file=/var/log/my.log &

Since the last changelogs, i need to remove (please confirm) :

-v because a new swith is coming --stats-log-level (awesome :))
–drive-full-list (obsolete?)

How works exactly the new switch(s), and what’s it does exactly ?

  --fast-list                           Use recursive list if available. Uses more memory but fewer transactions.
  --tpslimit float                    Limit HTTP transactions per second to this.
  --tpslimit-burst int              Max burst of transactions for --tpslimit. (default 1)

Any others advice to optimize a little bit the things?

Again, thanks alots for you works NCW it’s just AWESOME :slight_smile:

Please let me know…

2

+1
don’t know how to use these values
okay… values set to “1” still banned …

3

–fast-list won’t help with drive (it doesn’t support it), but try --tpslimit - that should help.

4

So that should be 3 for Google Drive?

5

@ncw even with --tpslimit 1 i’m getting banned… somehow there are always peaks up to 4 requests per second in my gdrive-dashboard … so it’s not working well… most of the time, but not always… which still results in bans

this is with tps-limit 1 … somehow there is still a peak up to 4 requests per second - and i got banned everytime …

34
Bildschirmfoto 2017-09-18 um 14.52.34.png932×562 11.8 KB

6

Hmm, I wonder what is causing those peaks. Are you running more than one copy of rclone maybe? I’m pretty sure the tps limiter is working in rclone, but rclone counts a transaction as an HTTP transaction - it is possible drive counts differently.

You can use --dump-headers to get an idea of when rclone does its http transactions - see if you can spot if it does them quicker than the tps set.