Googleapi: got HTTP response code 429

Help and Support
1

What is the problem you are having with rclone?

I make backups of my virtual servers and store them in google drive. I have been using this practice for a couple of years but since February I have been experiencing these problems on some backups that maybe larger in size than others. I did a test with a backup that I knew had not been uploaded to google drive. This one (called "albo") takes up 3.27GB. Other backups, which I am sure were uploaded to drive, on the other hand, take up less than 1.5 GB.

Run the command 'rclone version' and share the full output of the command.

rclone v1.53.3-DEV
os/arch: linux/amd64
go version: go1.18.1

Which cloud storage system are you using?

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

/opt/rclone copy /opt/backup/tmp/albo.unimol.it.giornaliero.tar.gz gdrive:unimol/backup/domini/host1

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

I would not want to update rclone so as not to cause problems with the scripts I have made that make periodic backups of my virtualservers.

A log from the command that you were trying to run with the -vv flag

Since the result of the command is long, I leave you the pastebin link:

2024/02/29 12:51:29 DEBUG : Creating backend with remote "/opt/backup/tmp/albo.unimol.it.giornaliero.tar.gz"
2024/02/29 12:51:29 DEBUG : Using config file from "/root/.rclone.conf"
2024/02/29 12:51:29 DEBUG : fs cache: adding new entry for parent of "/opt/backup/tmp/albo.unimol.it.giornaliero.tar.gz", "/opt/backup/tmp"
2024/02/29 12:51:29 DEBUG : Creating backend with remote "gdrive:unimol/backup/domini/host1"
2024/02/29 12:51:30 DEBUG : Google drive root 'unimol/backup/domini/host1': root_folder_id = "0AA-7VyoIoM4zUk9PVA" - save this in the config to speed up startup
2024/02/29 12:51:31 DEBUG : albo.unimol.it.giornaliero.tar.gz: Sizes differ (src 3521772559 vs dst 3521689684)
2024/02/29 12:51:32 DEBUG : albo.unimol.it.giornaliero.tar.gz: Sending chunk 0 length 8388608
2024/02/29 12:51:33 DEBUG : albo.unimol.it.giornaliero.tar.gz: Sending chunk 8388608 length 8388608
[........ Sending chunk ........]
2024/02/29 12:52:30 DEBUG : albo.unimol.it.giornaliero.tar.gz: Sending chunk 335544320 length 8388608
2024/02/29 12:52:31 INFO  :
Transferred:          328M / 3.280 GBytes, 10%, 5.500 MBytes/s, ETA 9m11s
Transferred:            0 / 1, 0%
Elapsed time:       1m1.7s
Transferring:
 *             albo.unimol.it.giornaliero.tar.gz:  9% /3.280G, 5.428M/s, 9m18s

2024/02/29 12:52:31 DEBUG : albo.unimol.it.giornaliero.tar.gz: Sending chunk 343932928 length 8388608
[........Sending chunk .........]
2024/02/29 12:52:50 DEBUG : albo.unimol.it.giornaliero.tar.gz: Sending chunk 452984832 length 8388608
2024/02/29 12:52:57 ERROR : albo.unimol.it.giornaliero.tar.gz: Failed to copy: googleapi:

Pastebin Log

The config file:

[gdrive]
type = drive
client_id = 
client_secret = 
token = xxxxxxxxxxxxxxxxxx
1 Like
2

Your version is is years OLD...

Please update and try again.

3

You might find this thread worth to read:

4

So should I try to update the version of rclone first? I read in the documentation that just use the command

rclone selfupdate

but unfortunately it is not available for this version. How should I do it without losing the configuration?

5

Download directly from Rclone downloads and put in some test folder.

You can keep both versions for testing.

If you worry about old configs then do not use them. Create copy.

6

Ok thanks, i'll try directly with this:

sudo -v ; curl https://rclone.org/install.sh | sudo bash

and i'll copy the config file so i'll keep it safe.

7

Not good idea... As this can overwrite your old version or create problems depending on your PATH settings.

Download directly.

If all working then uninstall your repo rclone version (using your OS packages manager) and then install rclone this way.

8

Thank you, I use Ubuntu 22.04.4 LTS (GNU/Linux 5.15.0-1052-oracle x86_64. I will try and let you know.

9

I followed the installation guide for linux that told me to do this:

Fetch and unpack

 curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip
 unzip rclone-current-linux-amd64.zipper
 cd rclone-*-linux-amd64

Copy binary file

 sudo cp rclone /usr/bin/
 sudo chown root:root /usr/bin/rclone
 sudo chmod 755 /usr/bin/rclone

Install manpage

 sudo mkdir -p /usr/local/share/man/man1
 sudo cp rclone.1 /usr/local/share/man/man1/
 sudo mandb

Run rclone config to setup. See rclone config docs for more details.

 rclone config

Having done this I left many fields blank in the configuration wizard. Then I copied the old config file replacing the new one found in /root/.rclone.conf
Correct?

10

if you want to use it as root then yes.

Does your problem still exist when you run new version?

11

I updated rclone:

rclone v1.65.2
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 5.15.0-1052-oracle (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.6
- go/linking: static
- go/tags: none

and:

[gdrive]
type = drive
client_id =
client_secret =
token = XXX
### Double check the config for sensitive info before posting publicly

but the i obtain the same error:

ERROR : Attempt 1/3 failed with 1 errors and: googleapi: got HTTP response code 429
12

What Google tells you:

googleapi: got HTTP response code 429
We're sorry...but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.

Clearly their system decided that they do not like your traffic... To get exact reason you would have to ask Google support.

Look at the thread I have linked earlier.

Suggested action was to use--drive-chunk-size=256M flag to send less chunks by increasing their size

13

This is the suggestion in the post you pointed me to:

I was able to solve that by using

  --drive-chunk-size=512M  \                                                                                                               
  --drive-pacer-burst=195  \                                                                                                               
  --drive-pacer-min-sleep=20000ms  \

So i can try to use this. For example, my command was:

/usr/bin/rclone copy /opt/backup/tmp/albo.unimol.it.giornaliero.tar.gz gdrive:(GDRIVE PATH)

With this new suggestion i need to do this instead (?):

/usr/bin/rclone copy --drive-chunk-size=512M /opt/backup/tmp/albo.unimol.it.giornaliero.tar.gz gdrive:(GDRIVE PATH)
14

Yes - give it a try.

15

Same error of google 429 :frowning:

16

We need gdrive expertise:)

@Animosity022 - do you have any suggestions here?

17

your computer or network may be sending automated queries. To protect our users, we can't process your request right now.
recently, there have been an increasing number of topics about this issue.
maybe gdrive has changed its policy or some change with rclone.

1 Like
18

Yeap. Something changed. But I tend to think it is on Google side - I remember somebody trying older rclone versions with the same results.

Maybe after killing unlimited nonsense Google now is trying to make sure that people use their services more in line with ToS? At the end if somebody needs more robust performance there is always GCS which I hear from friends and customers works great. Of course it cost more:)

19

In this thread, we had similar issues with GDrive and Hetzner, and adding a few DNS entries to the hosts file fixed it.
Maybe the same issue here too?

20

These are not free gdrive accounts - so the best way to clarify is to reach to google support asking for explanation. Then possibly rclone can accommodate what is needed.