Google Drive - can't refresh OAuth token in headless server

Help and Support
1

What is the problem you are having with rclone?

Oauth token cannot be refreshed on headless machine
I generated the token on Windows, same config file.
To get the original token I ran: rclone authorize drive --drive-scope drive.readonly,drive.file
The token refreshes there no problem. However, on the Linux headless machines (tested on RHEL7 and RHEL9), the refresh always fails.

Not sure if related, but one thing I noticed is that the refresh token has a different scope than the original token
Checking on https://www.googleapis.com/oauth2/v1/tokeninfo?access_token=
The original has both the drive.readonly,drive.file scopes, but the refresh one only has drive.file

Note: I switched to a service account for now, so I haven't tested this exhaustively

Run the command 'rclone version' and share the full output of the command.

rclone v1.65.2
- os/version: rocky 9.3 (64 bit)
- os/kernel: 5.14.0-362.18.1.el9_3.x86_64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.6
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync /path/to/src gdrive-oauth:dst -vv --dry-run

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[gdrive-oauth]
type = drive
client_id = XXX
client_secret = XXX
scope = drive.readonly,drive.file
root_folder_id = XXX
service_account_file =
team_drive =
token = XXX

[gdrive-svc]
type = drive
scope = drive.file,drive.metadata.readonly
root_folder_id = XXX
service_account_file = /home/rclone/.config/rclone/gdrive-svc.json
team_drive =

A log from the command that you were trying to run with the -vv flag

2024/01/28 18:29:41 DEBUG : rclone: Version "v1.65.2" starting with parameters ["/home/rclone/bin/rclone" "sync" "/path/to/src" "gdrive-oauth:dst" "-vv" "--dry-run"]
2024/01/28 18:29:41 DEBUG : Creating backend with remote "/path/to/src"
2024/01/28 18:29:41 DEBUG : Using config file from "/home/rclone/.config/rclone/rclone.conf"
2024/01/28 18:29:41 DEBUG : Creating backend with remote "gdrive-oauth:dst"
2024/01/28 18:29:41 DEBUG : gdrive-oauth: Loaded invalid token from config file - ignoring
2024/01/28 18:29:41 DEBUG : gdrive-oauth: got fatal oauth error: oauth2: "unauthorized_client" "Unauthorized"
2024/01/28 18:29:41 DEBUG : gdrive-oauth: Loaded invalid token from config file - ignoring
2024/01/28 18:29:41 DEBUG : gdrive-oauth: got fatal oauth error: oauth2: "unauthorized_client" "Unauthorized"
2024/01/28 18:29:41 DEBUG : gdrive-oauth: Loaded invalid token from config file - ignoring
2024/01/28 18:29:41 DEBUG : gdrive-oauth: got fatal oauth error: oauth2: "unauthorized_client" "Unauthorized"
2024/01/28 18:29:41 ERROR : Google drive root 'dst': error reading destination root directory: couldn't list directory: Get "https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Csha1Checksum%2Csha256Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks%2CresourceKey%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%27<ID>%27+in+parents%29+and+%28name%3D%27dst%27%29+and+%28mimeType%3D%27application%2Fvnd.google-apps.folder%27+or+mimeType%3D%27application%2Fvnd.google-apps.shortcut%27%29&supportsAllDrives=true": couldn't fetch token: unauthorized_client: if you're using your own client id/secret, make sure they're properly set up following the docs
2024/01/28 18:29:41 DEBUG : Google drive root 'dst': Waiting for checks to finish
2024/01/28 18:29:41 DEBUG : Google drive root 'dst': Waiting for transfers to finish
2024/01/28 18:29:41 ERROR : Google drive root 'dst': not deleting files as there were IO errors
2024/01/28 18:29:41 ERROR : Google drive root 'dst': not deleting directories as there were IO errors
2024/01/28 18:29:41 ERROR : Attempt 1/3 failed with 1 errors and: couldn't list directory: Get "https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Csha1Checksum%2Csha256Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks%2CresourceKey%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%27<ID>%27+in+parents%29+and+%28name%3D%27dst%27%29+and+%28mimeType%3D%27application%2Fvnd.google-apps.folder%27+or+mimeType%3D%27application%2Fvnd.google-apps.shortcut%27%29&supportsAllDrives=true": couldn't fetch token: unauthorized_client: if you're using your own client id/secret, make sure they're properly set up following the docs
2024/01/28 18:29:41 DEBUG : gdrive-oauth: Loaded invalid token from config file - ignoring
2024/01/28 18:29:41 DEBUG : gdrive-oauth: got fatal oauth error: oauth2: "unauthorized_client" "Unauthorized"
2024/01/28 18:29:41 ERROR : Google drive root 'dst': error reading destination root directory: couldn't list directory: Get "https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Csha1Checksum%2Csha256Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks%2CresourceKey%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%27<ID>%27+in+parents%29+and+%28name%3D%27dst%27%29+and+%28mimeType%3D%27application%2Fvnd.google-apps.folder%27+or+mimeType%3D%27application%2Fvnd.google-apps.shortcut%27%29&supportsAllDrives=true": couldn't fetch token: unauthorized_client: if you're using your own client id/secret, make sure they're properly set up following the docs
2024/01/28 18:29:41 DEBUG : Google drive root 'dst': Waiting for checks to finish
2024/01/28 18:29:41 DEBUG : Google drive root 'dst': Waiting for transfers to finish
2024/01/28 18:29:41 ERROR : Google drive root 'dst': not deleting files as there were IO errors
2024/01/28 18:29:41 ERROR : Google drive root 'dst': not deleting directories as there were IO errors
2024/01/28 18:29:41 ERROR : Attempt 2/3 failed with 1 errors and: couldn't list directory: Get "https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Csha1Checksum%2Csha256Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks%2CresourceKey%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%27<ID>%27+in+parents%29+and+%28name%3D%27dst%27%29+and+%28mimeType%3D%27application%2Fvnd.google-apps.folder%27+or+mimeType%3D%27application%2Fvnd.google-apps.shortcut%27%29&supportsAllDrives=true": couldn't fetch token: unauthorized_client: if you're using your own client id/secret, make sure they're properly set up following the docs
2024/01/28 18:29:41 DEBUG : gdrive-oauth: Loaded invalid token from config file - ignoring
2024/01/28 18:29:42 DEBUG : gdrive-oauth: got fatal oauth error: oauth2: "unauthorized_client" "Unauthorized"
2024/01/28 18:29:42 ERROR : Google drive root 'dst': error reading destination root directory: couldn't list directory: Get "https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Csha1Checksum%2Csha256Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks%2CresourceKey%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%27<ID>%27+in+parents%29+and+%28name%3D%27dst%27%29+and+%28mimeType%3D%27application%2Fvnd.google-apps.folder%27+or+mimeType%3D%27application%2Fvnd.google-apps.shortcut%27%29&supportsAllDrives=true": couldn't fetch token: unauthorized_client: if you're using your own client id/secret, make sure they're properly set up following the docs
2024/01/28 18:29:42 DEBUG : Google drive root 'dst': Waiting for checks to finish
2024/01/28 18:29:42 DEBUG : Google drive root 'dst': Waiting for transfers to finish
2024/01/28 18:29:42 ERROR : Google drive root 'dst': not deleting files as there were IO errors
2024/01/28 18:29:42 ERROR : Google drive root 'dst': not deleting directories as there were IO errors
2024/01/28 18:29:42 ERROR : Attempt 3/3 failed with 1 errors and: couldn't list directory: Get "https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Csha1Checksum%2Csha256Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks%2CresourceKey%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%27<ID>%27+in+parents%29+and+%28name%3D%27dst%27%29+and+%28mimeType%3D%27application%2Fvnd.google-apps.folder%27+or+mimeType%3D%27application%2Fvnd.google-apps.shortcut%27%29&supportsAllDrives=true": couldn't fetch token: unauthorized_client: if you're using your own client id/secret, make sure they're properly set up following the docs
2024/01/28 18:29:42 NOTICE:
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.2s

2024/01/28 18:29:42 DEBUG : 7 go routines active
2024/01/28 18:29:42 Failed to sync: couldn't list directory: Get "https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Csha1Checksum%2Csha256Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks%2CresourceKey%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%27<ID>%27+in+parents%29+and+%28name%3D%27dst%27%29+and+%28mimeType%3D%27application%2Fvnd.google-apps.folder%27+or+mimeType%3D%27application%2Fvnd.google-apps.shortcut%27%29&supportsAllDrives=true": couldn't fetch token: unauthorized_client: if you're using your own client id/secret, make sure they're properly set up following the docs
2

Update: I copied the token line from the Windows machine's config after one refresh, which had already expired.
Surprisingly, the token was refreshed without issues on the Linux server.
So it seems it can't refresh the original token for some reason on Linux (which on Windows it has no trouble refreshing... weird).
However, the scope from the refreshed token is still wrong, only having drive.file (though for my usage it's probably fine to only have that one).

3

You need to use the command that it gives you when you run rclone config - this contains the scope parameter encoded into the arguments. I think rclone authorize will just ignore the --drive-scope parameter.

This will be why it is failing - Google is fussy about that.

4

Thanks for the reply, I figured it out now.

You need to run the authorize command with the client_id and secret, such as:
rclone authorize drive --drive-scope <SCOPES> --drive-client-id <CLIENT_ID> --drive-client-secret <CLIENT_SECRET>

(alternatively, just run the config command on the machine with a web browser, and copy the contents of the config file to the headless machine)

I assumed the authorize command updated the config file on Windows (where I ran the command), so I was probably using another token, that's why it could be refreshed there and failed on Linux where I copied the token the authorize command gave me.

1 Like
5

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.