Google Drive Token refresh failed

Help and Support
#1

What is the problem you are having with rclone?

rclone mount eventually fails due to token refresh failure

What is your rclone version (output from rclone version)

rclone v1.54.0
- os/arch: linux/amd64
- go version: go1.15.

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Unraid 6.9.0-rc2

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone rcd --rc-no-auth --cache-dir /mnt/cache/rclone &
rclone rc mount/mount fs=gsuite: mountPoint=/mnt/remotes/gsuite mountOpt='{"AllowOther":true}' vfsOpt='{"CacheMode":3}'

The rclone config contents with secrets removed.

(this was after the rclone config reconnect mentioned below)

[gsuite]
type = drive
client_id = xxxx.apps.googleusercontent.com
client_secret = xxxx
scope = drive
root_folder_id = xxxx
service_account_file = 
token = {"access_token":"xxxx","token_type":"Bearer","refresh_token":"xxxx","expiry":"2021-02-12T22:20:38.978954814-05:00"}

A log from the command with the -vv flag

(this was from '--log-level DEBUG' not sure if different from '-vv'

2021/02/12 21:17:47 DEBUG : gsuite: Loaded invalid token from config file - ignoring
2021/02/12 21:17:47 DEBUG : gsuite: Token refresh failed try 1/5: context canceled
2021/02/12 21:17:48 DEBUG : gsuite: Loaded invalid token from config file - ignoring
2021/02/12 21:17:48 DEBUG : gsuite: Token refresh failed try 2/5: context canceled
2021/02/12 21:17:49 DEBUG : gsuite: Loaded invalid token from config file - ignoring
2021/02/12 21:17:49 DEBUG : gsuite: Token refresh failed try 3/5: context canceled
2021/02/12 21:17:50 DEBUG : gsuite: Loaded invalid token from config file - ignoring
2021/02/12 21:17:50 DEBUG : gsuite: Token refresh failed try 4/5: context canceled
2021/02/12 21:17:51 DEBUG : gsuite: Loaded invalid token from config file - ignoring
2021/02/12 21:17:51 DEBUG : gsuite: Token refresh failed try 5/5: context canceled
2021/02/12 21:17:52 INFO  : Google drive root '': Failed to get StartPageToken: Get "https://www.googleapis.com/drive/v3/changes/startPageToken?alt=json&prettyPrint=false&supportsAllDrives=true": couldn't fetch token - maybe it has expired? - refresh with "rclone config reconnect gsuite:": context canceled

I can fix with

rclone config reconnect gsuite:

but it will eventually fail again.

#2

See:

#3

I read over the other issue, but it's not clear what the actual solution is.

I checked my OAuth consent screen settings and it's set as "Production" publishing status and "External" user type already. I have never changed it so it has been that way since it was set up over 3 years ago.

I do see one difference in that I am using a GSuite account and my client ID is set as "Desktop" app type instead of "Other" as mentioned in the guide. There doesn't appear to be a way to change that.

Should I clear everything out and set up again from scratch?

#4

I used the same production key now since 2019 with no issues as I am production and "desktop" as well:

image
image1102×120 12.4 KB

#5

Yea my setup has been working fine for 3+ years until recently. Not sure what to do next.

In fact had to "rclone config reconnect gsuite:" three times today. I guess I will recreate from scratch and see if that helps.

#6

What's the full debug log look like when it pops up?

#7

Exactly as above with the 5 retries. It will continue repeating until performing the reconnect.

#8

More info. Sometimes it will fail to refresh within an hour or 2, but other times it seems to go all day without a failure. It seems like maybe when there is a lot of steady activity things work well, but if there is enough idle time is when the failure occurs.

In any case I created a new OAuth Client ID so we will see if that makes a difference.

#9

Token refresh failed 47 minutes later. :frowning:

#10 
2021/02/16 10:33:44 DEBUG : gsuite: Loaded fresh token from config file
2021/02/16 10:33:44 DEBUG : gsuite: Saved new token in config file

wonder if this mean it has healed itself?

#11

That is what it is supposed to look like - whenever rclone gets a new token it saves it in the config file.

#12

I wouldn't count on it.

I've been having a the same problem with a Google drive. Oddly it's only the drive associated with my account, shared drives seem to be stable.

#13

When I try to access my Google drive after letting it sit idle rclone reports the following errors.

2021/02/16 17:22:11 ERROR : Google drive root '': Statfs failed: failed to get Drive storageQuota: Get "https://www.googleapis.com/drive/v3/about?alt=json&fields=storageQuota&prettyPrint=false": couldn't fetch token - maybe it has expired? - refresh with "rclone config reconnect MyNHWG:": context canceled
2021/02/16 17:22:16 ERROR : /: Dir.Stat error: couldn't list directory: Get "https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%27root%27+in+parents%29&supportsAllDrives=true": couldn't fetch token - maybe it has expired? - refresh with "rclone config reconnect MyNHWG:": context canceled
2021/02/16 17:22:21 ERROR : /: Dir.Stat error: couldn't list directory: Get "https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%27root%27+in+parents%29&supportsAllDrives=true": couldn't fetch token - maybe it has expired? - refresh with "rclone config reconnect MyNHWG:": context canceled

This will repeat once or twice, several times if I'm trying to jump to a local mount point, or using the "Explorer" in the web gui. Things come back after issuing a rclone reconnect command and going through the refresh process:

> rclone config reconnect MyDrive:/

Oddly all of the shared drives I have connected stay very stable.

#14

Try commenting out client_id and client_secret in the config file to use rclone's default ones. Then do an rclone config reconnect and see if you get the same problem?

This will tell us whether it is a property of your client_id or not.

#15

Eventually failed to refresh again with my client_id. I removed as @ncw suggested and will report back how it goes.

1 Like
#16

Token refresh failed again, exact same sequence in log.

#17

Hmm...

Is there anything unusual about your rclone activity? Might you be running multiple rclone's at the same time? Or have you copied the config file many times to other machines?

#18

Since yesterday I happened to catch it just as it repeated the refresh failure and retry 5 times, but then it recovered and was able to refresh the token on the next minute.

I'm not doing anything differently than I have done for years. I use an rclone mount and mergerfs merging with local files into a single source that feeds Plex/Sonarr/Radarr. New files are pushed to Google Drive through a separate "rclone copy" so that I can apply the --bwlimit for uploads.

I did recently move everything from an old Ubuntu machine to a new Unraid machine and did copy over the rclone config. However, that had been running with no trouble for a few months before this issue started occurring.

#19

I'm having the same problem:

Failed to sync: couldn't list directory: Get "<>": couldn't fetch token - maybe it has expired? - refresh with "rclone config reconnect ID:": oauth2: cannot fetch token: 400 Bad Request
Response: {
"error": "invalid_grant",
"error_description": "Bad Request"
}

Updating the token is impossible for me, it is a script that runs local backup of 90 team drives 1x per week, it is difficult to update 90 tokens .....

I contacted google support and they reported that the problem is that the tokens being generated by RClone have very short validity, I believe they are 0auth and now the google API uses 0auht2

Does anyone know how to make the token's validity get longer?

#20

In the message you posted, you can see it uses oauth2 :slight_smile:

Tokens last one hour per Google's API as that's nothing to do with rclone. It doesn't sound like you got a very good response from them about the issue unfortunately as they gave you a run around.