GDrive – Seeing errors in API dashboard

[nle]

Hi, I’m seeing errors in Googles API interface.

I have increased the API limit from Google to the max they allow:

The command I’m using at 2 am every day:

rclone sync /mnt/path/to/backup gdrive-crypt:/last_snapshot --backup-dir=gdrive-crypt:/archive/2018/2018-08-28_02:00:01 --log-file=/path/to/log/_rclone/log/rclone_jobber.log --log-level=INFO --config /path/to/config/_rclone/rclone.conf --filter-from=/path/to/config/_rclone/filter_rules --transfers=32 --checkers=16 --drive-chunk-size=256M --modify-window=1s

rclone is reporting no problems, and the last 14 days I have not really uploaded anything.

Transferred:   1.665 kBytes (0 Bytes/s)
Errors:                 0
Checks:            176806
Transferred:            1
Elapsed time:   3h11m8.8s

Version:

rclone v1.42
- os/arch: linux/amd64
- go version: go1.10.1

Any input on this?

(I noticed this because I’m trying to use gphotosuploader to upload images to Google Photos, but keep seeing can't get an upload url and it wont upload anything. A bug report with the same issue claims that the message indicates you are “blacklisted” by Google)

[Animosity022]

Can you post a screenshot of your queries per 100 seconds?

It should look like this:

You are doing 32 transfers at a time, which is going to generate a lot of API errors as you can only really sustain:

Which is at most 10 per second for 100 seconds.

I’d way back off the transfers and move that to 5 along with the checkers Mine is mainly for large single files and I only move 10-20 files a night:

/usr/bin/rclone move /data/local/ gcrypt: --checkers 3 --fast-list --log-file /home/felix/logs/upload.log -v --tpslimit 3 --transfers 3 --exclude-from /home/felix/scripts/excludes --delete-empty-src-dirs

[nle]

Thanks, yes that makes sense. I think I got my numbers from another post where someone checked what gave them the most juice per upload.

My files vary alot, could be small or big. Many or few.

Here is the screenshot:

[Animosity022]

Yeah, your per 100 doens’t look bad either.

I’m guessing you are hitting the limit hard and it’s throttling you. Try smaller and work your way up to see what a good spot is.

[nle]

No, thats the thing. I don’t feel or notice that I have any issues with rclone, but the API console shows errors.
I’m just wondering why.

[Animosity022]

If you turn up to debug, you’ll see a bunch of errors / rate limiting and such. rclone handles that and backs off and retries, but no reason to get the errors.

You’ll get an overall better experience if you tune it a little down to alleviate the errors.

It’s like if you are filling a bottle without a funnel and just splashing water everywhere. The bottle gets fillled, but you get water everywhere. Better to get done faster without the splashing

[nle]

Yes, you are right. I’ll look into that.

Appreciate the input.