New flag for google drive --drive-stop-on-upload-limit to stop at 750 GB limit

In the latest beta v1.50.2-131 or later I've added this flag. I think this should work but testing appreciated by heavy uploaders :slight_smile:

--drive-stop-on-upload-limit

Make upload limit errors be fatal

At the time of writing it is only possible to upload 750GB of data to
Google Drive a day (this is an undocumented limit). When this limit is
reached Google Drive produces a slightly different error message. When
this flag is set it causes these errors to be fatal. These will stop
the in-progress sync.

Note that this detection is relying on error message strings which
Google don't document so it may break in the future.

See: https://github.com/rclone/rclone/issues/3857

  • Config: stop_on_upload_limit
  • Env Var: RCLONE_DRIVE_STOP_ON_UPLOAD_LIMIT
  • Type: bool
  • Default: false
3 Likes

Remember: Google counts normal uploads & server side side copy separately. So you can do 750gb normal uploads + 750gb server side copy a day.

Also note, Google starts to throw quota errors from 748 Gb, then slowly increases the errors if you try to upload more.

Google doesn’t mind if you try to upload 5-6 small files even if you reach your daily quota.

1 Like

Using rclone v1.50.2-146-g3a1b41ac-beta doesn't work, it stills show this message and continues
2020/01/18 22:09:07 ERROR : Path/to/folder: error reading destination directory: couldn't list directory: googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=XXXXXXXXXXXX, userRateLimitExceeded

Can you run the full command now with the flag and -vv on and share the output?

The error you have above is a normal slow down message in terms of listing a directory and does not appear to be related to the upload quota.

You still haven't reached the capped state according to our tests. See the issue here for what the expected error message is when you get capped at 750 GB.

A few of the 403 messages are normal BTW.

Quick testing:

Works with non-server-side uploads? Yes
Works with server-side copy/sync? Yes
Works when account has full 750GB quota remaining? Yes
Works when account has partial quota (100-300GB) remaining? Yes
Works when account has zero quota remaining? Yes

@ncw Thank you! Outstanding addition :+1:

:slight_smile: Thanks for testing! I think this flag is quite likely to get broken by changes at Google so we'll need to keep an eye on it!