Gdrive: "Error 403: User rate limit exceeded" For more than a week

Help and Support
1

What is the problem you are having with rclone?

I am getting the following error message for more than a week now:
ERROR : Google drive root '_decrypt': Received upload limit error: googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded
I am not able to sync my data from one Gdrive to another.
At first i thought i have a problem with the daily 750 GB contingent, but first I have the "--max-transfer 745G" parameter and second I am not able to sync more than a week now.
I have created and set up a new Client-IDs & Tokens for myself several times. Nothing helped.
Do you guys think that there is a longer ban on the part of Google?

What is your rclone version (output from rclone version)

rclone v1.53.4 and also tested with the latest beta release

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Ubuntu Server 20.04

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

/usr/bin/rclone sync gdrive1: gdrive2: --config /home/XXX/.config/rclone/rclone.conf --log-file /home/XXX/google_cloud_transfer_job.log -vv --max-transfer 745G --drive-server-side-across-configs --error-on-no-transfer --cutoff-mode=cautious --create-empty-src-dirs --drive-stop-on-upload-limit --auto-confirm --drive-acknowledge-abuse --max-duration 1h --stats-file-name-length 0

The rclone config contents with secrets removed.

[Gdrive1]
type = drive
client_id = XXXXXXXXXXXXXXXXXXXX.apps.googleusercontent.com
client_secret = XXXXXXXXXXXXXXXXXXXX
scope = drive
token = {"access_token":"XXXXXXXXXXXXXXXXXXXX","token_type":"Bearer","refresh_token":"XXXXXXXXXXXXXXXXXXXX","expiry":"2021-01-31T18:07:02.337430616+01:00"}
team_drive = XXXXXXXXXXXXXXXXXXXX
root_folder_id =
server_side_across_configs = true

[Gdrive1_crypt]
type = crypt
remote = Gdrive1:
filename_encryption = standard
directory_name_encryption = true
password = XXXXXXXXXXXXXXXXXXXX

[Gdrive2]
type = drive
client_id = XXXXXXXXXXXXXXXXXXXX.apps.googleusercontent.com
client_secret = XXXXXXXXXXXXXXXXXXXX
scope = drive
token = {"access_token":"XXXXXXXXXXXXXXXXXXXX","token_type":"Bearer","refresh_token":"XXXXXXXXXXXXXXXXXXXX","expiry":"2021-01-31T18:07:09.666137375+01:00"}
team_drive = XXXXXXXXXXXXXXXXXXXX
root_folder_id =
server_side_across_configs = true

[Gdrive2_crypt]
type = crypt
remote = Gdrive2:
filename_encryption = standard
directory_name_encryption = true
password = XXXXXXXXXXXXXXXXXXXX

A log from the command with the -vv flag

2021/01/31 17:25:42 DEBUG : rclone: Version "v1.53.4" starting with parameters ["/usr/bin/rclone" "sync" "Gdrive1:" "Gdrive2:" "-vvv" "--max-transfer" "745G" "--drive-server-side-across-configs" "--error-on-no-transfer" "--cutoff-mode=cautious" "--create-empty-src-dirs" "--drive-stop-on-upload-limit" "--auto-confirm" "--drive-acknowledge-abuse" "--max-duration" "1h" "--stats-file-name-length" "0"]
2021/01/31 17:25:42 DEBUG : Using config file from "/home/XXX/.config/rclone/rclone.conf"
2021/01/31 17:25:42 DEBUG : Creating backend with remote "Gdrive1:"
2021/01/31 17:25:42 DEBUG : Creating backend with remote "Gdrive2:"
2021/01/31 17:25:42 INFO  : Google drive root '': Transfer session deadline: 2021/01/31 18:25:42
2021/01/31 17:25:42 DEBUG : Gdrive2: Loaded invalid token from config file - ignoring
2021/01/31 17:25:42 DEBUG : Gdrive2: Saved new token in config file
2021/01/31 17:25:49 ERROR : Google drive root '': Received upload limit error: googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded
2021/01/31 17:25:49 ERROR : a9fmpfn4jle1r2790q93oqqc2c/4sibv9pou760ru2vlril2k7uh4/mu7c0sql2amk6g87b1f6qjjqq8/0f6n3dngmn2l2fa90gfdlqflq9ce3n763gbik9kmbgv0ijk030qm2livniufeqn4g7k1dm2b3vs3ph3uhhi29qrs1jtgcgdu67395u8: Failed to copy: googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded
2021/01/31 17:25:49 ERROR : Cancelling sync due to fatal error: googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded
2021/01/31 17:25:49 ERROR : a9fmpfn4jle1r2790q93oqqc2c/1qieb6hvdglliik06rcuq60f0g/8bcosffkvpp3bd9df0jla852os: error reading source directory: couldn't list directory: context canceled
2021/01/31 17:25:49 ERROR : a9fmpfn4jle1r2790q93oqqc2c/eouoa0245l07ievkunoeltccv8/mu7c0sql2amk6g87b1f6qjjqq8: error reading source directory: couldn't list directory: context canceled
2021/01/31 17:25:49 ERROR : a9fmpfn4jle1r2790q93oqqc2c/hvravqd022pokp7gherhqskm30: error reading source directory: couldn't list directory: context canceled
2021/01/31 17:25:49 ERROR : a9fmpfn4jle1r2790q93oqqc2c/g1q70ffplo4h521l1m96jqi15o/mu7c0sql2amk6g87b1f6qjjqq8: error reading source directory: couldn't list directory: context canceled
2021/01/31 17:25:49 ERROR : a9fmpfn4jle1r2790q93oqqc2c/0jpj2shlrq9n3rs0snpnvepqs0dmm45bd9bn393r4di3sr22a9h0/i9ah1d1cpdmlufs9ihlbv9pvk8: error reading source directory: couldn't list directory: context canceled
2021/01/31 17:25:49 ERROR : a9fmpfn4jle1r2790q93oqqc2c/h4jbumbslliifjbngvvn07r20c/40pbnnl4nr05sdggk0932q09a8: error reading source directory: couldn't list directory: context canceled
2021/01/31 17:25:49 ERROR : a9fmpfn4jle1r2790q93oqqc2c/9ji1ief9tohifdjj6aaoa6jn47ioqmplf0t85njfprdm1j3br55g/qni5ceje6ncoe716d9ver466fg: error reading source directory: couldn't list directory: context canceled
2021/01/31 17:25:49 ERROR : a9fmpfn4jle1r2790q93oqqc2c/h110mo4u8u8p8kamthh72gamk8/4o50l76h2r5pkf3bqr04v2r1vg: error reading source directory: couldn't list directory: context canceled
...
...
...
2021/01/31 17:28:24 DEBUG : Google drive root '': failed to copy 393 directories
2021/01/31 17:28:24 ERROR : Google drive root '': not deleting files as there were IO errors
2021/01/31 17:28:24 ERROR : Google drive root '': not deleting directories as there were IO errors
2021/01/31 17:28:24 INFO  : There was nothing to transfer
2021/01/31 17:28:24 ERROR : Fatal error received - not attempting retries
2021/01/31 17:28:24 INFO  :
Transferred:             0 / 2.337 TBytes, 0%, 0 Bytes/s, ETA -
Errors:               393 (fatal error encountered)
Transferred:            0 / 761, 0%
Elapsed time:        34.5s

image
image2311×871 131 KB

2

Have you checked your source? Because I don't think the problem came from Google.

3

Hi @Hoang_Nguyen,

thank you for your reply. After checking the source I found out that I am not able to download files from my source files, this seems to last since more than 1 week now.
Has somebody experience with those bans from Google site?

  • How long does this last?
  • What could have caused it?
  • Does a ban extend if I try to sync again?
4

I'm experiencing same issue, started also about week ago. I can download and upload files for both drives, but can't sync them

5

I removed
--drive-server-side-across-configs
from my script and it's working correctly, but of course sync take much longer time. So i guess there's issue with server side copying. Any solution?

6

Best to use the help and support template to capture all the information.

If it's working without it, you are hitting quota on the server side possibly as a guess. None of those items are documented.

7

I'm in contact with Modjo and can clarify things up a little:

  • First thing we found out is, that the source-files gave an error indicating the DownloadQuota has exceeded when downloading via drive.google.com - Download was tried using a external user with administrator privileges - after a few days keeping the files untouched, the error message did NOT change. A further investigation hinted us to the phenomenon @ryhoo describes as well

  • The --server-side-across-configs - flag was not working anymore (it did in the past on the 2 shares) resulting in a 403 Error UploadQuotaExceeded - error - without the flag it was possible to sync, but only with the user, who is part of the business-account.

  • After using a second user with Administrator-permissions, the files where not able to sync either way. It seems only the Owner of the File is able to download it as well. As result we came up with the final statement: rclone sync srcUsedByOwnerOfShare: dstNewShare:

  • A misleading trail was the ERROR, rclone -vv was giving - it throws a 403 Upload-Error, when the Download of the Source is not possible

  • Regarding the fact, that the --server-side-across-configs worked in the past, we can only speculate, that the businessowner of the teamdrive activated some kind of setting, which denys users from outside the business to download files, even when the permission is set by the user via drive.google.com

Maybe it would be possible, to alter the ERROR-Message to react on this kind of issues

8

Can you share a debug log with the error you are describing?

9

It is already in the starting post :wink:

The error messages I described are the ones offered by the drive.google.com WebIF, when trying to download these files.

10

I see as Google Drive is returning 403 user rate limit error and you are referencing the text from rclone not from Google's API as I thought you were mentioning a new / different 403 error from Google's API that we could look for.

Rclone only knows what the API is reporting based on what is being executed as it's trying to upload and the error message returned from Google is such as I'd say the error is not misleading but exactly what's being reported.

11

This is the problem - the 403 UploadQuota was not right at this point - the true error was kind of "Unable to download from source" - rclone raises 403 Upload, but I think this arises, as the server-side-sync is not able to read the source - in terms of debugging the error misleads to to asumption, that we hit the uploadQuota, but this was not right, as the UploadQuota was fine at this moment...

12

I get your point, but the problem is rclone only reports what it sees.

It tries to copy and that was the error the API reported back.

Further troubleshooting leads to the root cause.

My point being is rclone can only return what the API returns so misleading or not, that's what rclone sees.

13

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.