I have 36403 files I am trying to get backed up into my Google Drive. I have followed the steps at https://rclone.org/drive/ (under “Making your own client_id”) to make my own client ID and client secret. However, I never seem to get more than about 30% of the way through the backup set before rclone starts giving me 401 errors:
Response: {
"error" : "invalid_client",
"error_description" : "The OAuth client was not found."
}
If I login to the Google API console, I can still see the client in there. If I do rclone config and ask it to generate a new token - WITHOUT saying yes to editing the client ID and client secret - it will generate a token URL (this is a headless box) that gives me an HTTP 400 error.
If I edit the client ID and client secret, and simply copy/paste the previous values as displayed by rclone, the token URL will work. I can then issue rclone -v copy /mnt/zpool1/business crypt:/business, and it will run fine for a few hours, and then repeats the whole process from the beginning.
I can totally understand that I’m probably hitting some Google Rate Limit, but in my case I have the ownership of the Google Business account being used, so I could create more users/API keys, if rclone were able to round-robin between them, thus preventing me hitting the API limit. However, I can’t find any indication that this is a possible option; also, it doesn’t really explain to me why I have to issue an rclone config and fake rclone into thinking I am editing the client ID and client secret in order to get a new, valid token. Can anyone help me understand what I’m doing wrong there? Also, is there any plan or mechanism in place to leverage multiple client IDs to get past the API rate limits?
chrony is configured on the box and seems to be working correctly (if the clock is off, it’s by less than a full second); however, when looking through the logs with -vv, I cannot see anything that even LOOKS like an attempt to refresh the token. I have generated a new token and am running it with -vv now to share with you.
I changed "expiry":"2017-08-02T21:16:21.169188519-06:00" to "expiry":"2015-08-02T21:16:21.169188519-06:00", and it immediately returns a 401. Sent you a private message with a link to the logs (using -vv --dump-headers --log-file).
that is strange, it just says the client_id is invalid. Can you check the config file at this point after rclone has stopped to see if the client_id and client_secret look OK in it?
The client_id and secret in .rclone.conf never appear to be changed - in fact, the method I have for “fixing” the problem is to run rclone config and copy/paste the values it shows. When prompted to choose whether or not I need to edit the client_id and client_secret, if I say “No”, the token refresh link that is generated gives a 400 error in Google. However, if I edit the values and just REPASTE the exact value, it generates a valid token URL and everything starts working again (until the next timeout).
Can you see what changes before and after in the config file when you do the re-authenticate?
Any chance you could do that again but with --dump-bodies -vv? This will definitely have confidential stuff in though! Maybe you could examine it to see if the client_id is correct in the POST.
Using --dump-bodies showed there was an encoded null byte at the end of the client_id line; I deleted the entire .rclone.conf and regenerated it, and it has been running for about 18 hours now with no problem. No idea where the null byte came from or how it kept coming back through copy/paste, but thanks for sticking with me and showing me how to solve it.
