Gsuite 403 downloadQuotaExceeded

Help and Support
1

Hey everyone,

I started to have 403 errors “downloadQuotaExceeded” after copying one gdrive to another and possibly hitting the 750GB upload quota. I thought I had a 24h ban but even 2 days after the same problem was occuring.

After about 3 days (sorry can’t be more precise), I could download my files again though my download speed where only 1.5MB instead of 75MB. Using the v2 flag improved the speeds significantly.

The same downloadQuotaExceeded showed up the next day, although I am pretty sure I have not hit the 750GB quota. Creating new OAuth credentials and editing my rclone config seem to have resolved the problem for now, though I still don’t understand what happened.

Anybody met the same issues?

2

I know the debug logs are you huge, but it would be very helpful to see the logs around when the error occurs from the copying.

I think that would help the folks identify the problem as you should not see that type of issue.

3

i am experiencing the same thing.
Read: read=0, err=open file failed: googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded - on all files i tested

never had this before, v3 & v2 drive api, no huge uploads/downloads in the last 24h - very strange and worrying…

4

I am having the same issues. Started about 4 or 5 days ago. I have tried every adjustment possible and nothing works. It definitely appears Google has changed something.

5

@Animosity022 This is one log from this morning with my old OAuth:

rclone copy eplex_upload_unltd:tv/Game\ of\ Thrones/Season03/Game_of_Thrones-S03E05-Kissed_by_FireBluRay_720p_5.1.mkv eplex_upload_edu:test -vv
2018/10/16 11:37:36 DEBUG : rclone: Version “v1.44” starting with parameters [“rclone” “copy” “eplex_upload_unltd:tv/Game of Thrones/Season03/Game_of_Thrones-S03E05-Kissed_by_FireBluRay_720p_5.1.mkv” “eplex_upload_edu:test” “-vv”]
2018/10/16 11:37:36 DEBUG : Using config file from “/home/shenzo/.config/rclone/rclone.conf”
2018/10/16 11:37:37 DEBUG : remote_unltd: Saved new token in config file
2018/10/16 11:37:41 DEBUG : pacer: Rate limited, sleeping for 1.107621472s (1 consecutive low level retries)
2018/10/16 11:37:41 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/10/16 11:37:41 DEBUG : pacer: Rate limited, sleeping for 2.427011286s (2 consecutive low level retries)
2018/10/16 11:37:41 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/10/16 11:37:42 DEBUG : pacer: Rate limited, sleeping for 4.575453478s (3 consecutive low level retries)
2018/10/16 11:37:42 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/10/16 11:37:45 DEBUG : pacer: Rate limited, sleeping for 8.440978856s (4 consecutive low level retries)
2018/10/16 11:37:45 DEBUG : pacer: low level retry 4/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/10/16 11:37:49 DEBUG : pacer: Resetting sleep to minimum 10ms on success
2018/10/16 11:37:58 DEBUG : remote_edu: Saved new token in config file
2018/10/16 11:38:01 DEBUG : pacer: Rate limited, sleeping for 1.066954985s (1 consecutive low level retries)
2018/10/16 11:38:01 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/10/16 11:38:01 DEBUG : pacer: Resetting sleep to minimum 10ms on success
2018/10/16 11:38:02 DEBUG : pacer: Rate limited, sleeping for 1.938025525s (1 consecutive low level retries)
2018/10/16 11:38:02 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/10/16 11:38:02 DEBUG : pacer: Rate limited, sleeping for 2.628468623s (2 consecutive low level retries)
2018/10/16 11:38:02 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/10/16 11:38:04 DEBUG : pacer: Rate limited, sleeping for 4.874294364s (3 consecutive low level retries)
2018/10/16 11:38:04 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/10/16 11:38:07 DEBUG : pacer: Resetting sleep to minimum 10ms on success
2018/10/16 11:38:12 DEBUG : Game_of_Thrones-S03E05-Kissed_by_FireBluRay_720p_5.1.mkv: Couldn’t find file - need to transfer
2018/10/16 11:38:12 ERROR : Game_of_Thrones-S03E05-Kissed_by_FireBluRay_720p_5.1.mkv: Failed to copy: failed to open source object: open file failed: googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded
2018/10/16 11:38:12 ERROR : Attempt 1/3 failed with 1 errors and: failed to open source object: open file failed: googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded
2018/10/16 11:38:12 DEBUG : Game_of_Thrones-S03E05-Kissed_by_FireBluRay_720p_5.1.mkv: Couldn’t find file - need to transfer
2018/10/16 11:38:12 ERROR : Game_of_Thrones-S03E05-Kissed_by_FireBluRay_720p_5.1.mkv: Failed to copy: failed to open source object: open file failed: googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded
2018/10/16 11:38:12 ERROR : Attempt 2/3 failed with 1 errors and: failed to open source object: open file failed: googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded
2018/10/16 11:38:13 DEBUG : Game_of_Thrones-S03E05-Kissed_by_FireBluRay_720p_5.1.mkv: Couldn’t find file - need to transfer
2018/10/16 11:38:13 ERROR : Game_of_Thrones-S03E05-Kissed_by_FireBluRay_720p_5.1.mkv: Failed to copy: failed to open source object: open file failed: googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded
2018/10/16 11:38:13 ERROR : Attempt 3/3 failed with 1 errors and: failed to open source object: open file failed: googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded
2018/10/16 11:38:13 Failed to copy: failed to open source object: open file failed: googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded

6

@Animosity022 This is my log 3 hours after, after creating new OAuth and reconfiguring my rclone remotes:

2018/10/16 13:10:56 INFO : Game_of_Thrones-S03E05-Kissed_by_FireBluRay_720p_5.1.mkv: Copied (new)
2018/10/16 13:10:56 INFO :
Transferred: 2.648G / 2.648 GBytes, 100%, 24.387 MBytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 1 / 1, 100%
Elapsed time: 1m51.1s

2018/10/16 13:10:56 DEBUG : 7 go routines active
2018/10/16 13:10:56 DEBUG : rclone: Version “v1.44” finishing with parameters [“rclone” “copy” “eplex_upload_unltd2:tv/Game of Thrones/Season03/Game_of_Thrones-S03E05-Kissed_by_FireBluRay_720p_5.1.mkv” “eremote_edu:test” “-vv”]

You guys should try creating new OAuth and see if that changes anything. Still does not answer what is actually happening.

7

same thing after reauthenticating for new oauth credentials :frowning:

8

@seuffert Did you paramater it correctly when creating a new remote? I don’t think it works if you don’t enter clientID and secret manually as it may go with the default one.

9

I think if I’m not mistaken, there is a different limit if you are doing GDrive to GDrive, which is a server side copy.

I wonder if you are hitting that? I’m also on 1.44 currently and last night as an example uploaded 200GB with 0 errors in any logs.

Just for completeness, can you share your rclone.conf as well with the password/keys out?

10

Is that from a mount or from using an rclone copy?

11

yes i did, it is setup with my own client credentials

12

the log message is from a mount

13

Are you using a cache backend as well?

14

not using cache backend:

   rclone mount gd: /tmp/gd \
            -vv --stats 30s --read-only --allow-other --rc --umask 0222 --attr-timeout 1s \
            --dir-cache-time 672h --vfs-cache-max-age 675h \
            --vfs-read-chunk-size 8M \
            --buffer-size 0 \
            --poll-interval 1m0s \
            --drive-v2-download-min-size 0
15

I’m not using (never have) the v2 flags. Based on the other posts, it really does seem something with that.

16

the v2 api makes no difference, with or without the result is still the same

17

I mean, if you got the ban, you’d have to wait for it to lift. Are you saying that you got the messages after waiting ~24 hours without the v2 flag?

18

no , the messages appeared today. the strange part is, that in the last 24h there was very little api activity according to the api console, and not a lot downloads either

19

I am doing gdrive1 to gdrive2 though. Server side copy only works within one gdrive, no? I am pretty sure I could copy x00GB now that the 403 error does not pop up anymore. Will test tonight.

Here’s my rclone.conf, you can see I have basically duplicated remote_unltd: to remote_unltd2: except that I have used the new credentials. Not sure what else you can get out of it:

[remote_edu]
type = drive
client_id =
client_secret =
token =

[remote_unltd]
type = drive
client_id =
client_secret =
token =

[eplex_upload_unltd]
type = crypt
remote = remote_unltd:/Plex/
filename_encryption = standard
password =
password2 =

[eplex_upload_edu]
type = crypt
remote = remote_edu:/Plex/
filename_encryption = standard
password =

[remote_unltd2]
type = drive
client_id =
client_secret =
scope = drive
v2_download_min_size = 1G
token =

[eplex_upload_unltd2]
type = crypt
remote = remote_unltd2:/Plex/
filename_encryption = standard
password =
password2 =

20

I really thing something has changed on Google side. I actually have been using node-gdrive-fuse and have been for almost 2 years with absolutely no issues. I then started getting bans and first figured that since this code is so old, it must be the problem. I promptly switched to rclone with a cached drive and all. Got banned again. I even started getting paranoid that my google drive account was somehow compromised and people were downloading without my knowledge so I changed all api keys/clients/secrets and changed my password. Still got banned. I know for a fact that I am WAY under the query quotas shown in the api console. And I think from the time I got unbanned to being re-banned, A total of 3-4 movies were played in plex. shrugs