Invalid Credentials, authError

What is the problem you are having with rclone?

Getting the following error:
2021/01/27 10:33:22 Failed to sync: couldn't list directory: googleapi: Error 401: Invalid Credentials, authError

I thought I had fixed it by creating new Credentials in the Google Drive API Console but a week later it comes back.

The only thing that stands out is this:

{"access_token":"access_token","token_type":"Bearer","refresh_token":"1//refresh_token","expiry":"0001-01-01T00:00:00Z"}

How come the expiry is not a future date? The date is correct on my server.

Any help would be appreciated!

What is your rclone version (output from rclone version)

rclone v1.53.3
- os/arch: linux/amd64
- go version: go1.15.5

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Debian Buster
Linux server.home.lan 5.4.78-2-pve #1 SMP PVE 5.4.78-2 (Thu, 03 Dec 2020 14:26:17 +0100) x86_64 GNU/Linux

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync --progress --max-age=25h --tpslimit=3 --transfers=3 --checkers=3 --drive-chunk-size=512M --backup-dir=encrypted_daily:rlcone_trash/`date +"%Y-%m-%d-%T"` /mnt/volume2/data/ encrypted_daily:/data/

The rclone config contents with secrets removed.

[gdrive]
type = drive
client_id = client_id.apps.googleusercontent.com
client_secret = secret
scope = drive
token = {"access_token":"access_token","token_type":"Bearer","refresh_token":"1//refresh_token","expiry":"0001-01-01T00:00:00Z"}
root_folder_id = 0AMeNMJVetPmSUk9PVA

[encrypted_daily]
type = crypt
remote = gdrive:encrypted_daily
filename_encryption = standard
directory_name_encryption = true
password = passworf
password2 = password2

A log from the command with the -vv flag

root@server:~# rclone sync --progress --max-age=25h --tpslimit=3 --transfers=3 --checkers=3 --drive-chunk-size=512M --backup-dir=encrypted_daily:rlcone_trash/`date +"%Y-%m-%d-%T"` /mnt/volume2/data/ encrypted_daily:/data/ -vv
2021/01/27 11:07:10 DEBUG : --max-age 1.0416666666666667d to 2021-01-26 10:07:10.992540646 -0700 MST m=-89999.972020307
2021/01/27 11:07:10 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "sync" "--progress" "--max-age=25h" "--tpslimit=3" "--transfers=3" "--checkers=3" "--drive-chunk-size=512M" "--backup-dir=encrypted_daily:rlcone_trash/2021-01-27-11:07:10" "/mnt/volume2/data/" "encrypted_daily:/data/" "-vv"]
2021/01/27 11:07:10 DEBUG : Creating backend with remote "/mnt/volume2/data/"
2021/01/27 11:07:10 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2021/01/27 11:07:10 INFO  : Starting HTTP transaction limiter: max 3 transactions/s with burst 1
2021/01/27 11:07:10 DEBUG : Creating backend with remote "encrypted_daily:/data/"
2021/01/27 11:07:11 DEBUG : Creating backend with remote "gdrive:encrypted_daily/h0ck5ac0nfad6poklaqiebi01s"
2021-01-27 11:07:11 DEBUG : Creating backend with remote "encrypted_daily:rlcone_trash/2021-01-27-11:07:10"
2021-01-27 11:07:11 DEBUG : Creating backend with remote "gdrive:encrypted_daily/mescolv4avjnc57lhke1neiqrg/sbsea1liqgaofvv8cvtm10qrbnpemo72ka73meord94pqipohs8g"
2021-01-27 11:07:12 DEBUG : vzdump-qemu-100-2020_07_03-17_20_59.vma.gz: Excluded
2021-01-27 11:07:12 ERROR : : error reading destination directory: couldn't list directory: googleapi: Error 401: Invalid Credentials, authError
2021-01-27 11:07:12 DEBUG : Encrypted drive 'encrypted_daily:/data/': Waiting for checks to finish
2021-01-27 11:07:12 DEBUG : Encrypted drive 'encrypted_daily:/data/': Waiting for transfers to finish
2021-01-27 11:07:12 ERROR : Encrypted drive 'encrypted_daily:/data/': not deleting files as there were IO errors
2021-01-27 11:07:12 ERROR : Encrypted drive 'encrypted_daily:/data/': not deleting directories as there were IO errors
2021-01-27 11:07:12 INFO  : There was nothing to transfer
2021-01-27 11:07:12 ERROR : Attempt 1/3 failed with 1 errors and: couldn't list directory: googleapi: Error 401: Invalid Credentials, authError
2021-01-27 11:07:12 DEBUG : vzdump-qemu-100-2020_07_03-17_20_59.vma.gz: Excluded
2021-01-27 11:07:12 ERROR : : error reading destination directory: couldn't list directory: googleapi: Error 401: Invalid Credentials, authError
2021-01-27 11:07:12 DEBUG : Encrypted drive 'encrypted_daily:/data/': Waiting for checks to finish
2021-01-27 11:07:12 DEBUG : Encrypted drive 'encrypted_daily:/data/': Waiting for transfers to finish
2021-01-27 11:07:12 ERROR : Encrypted drive 'encrypted_daily:/data/': not deleting files as there were IO errors
2021-01-27 11:07:12 ERROR : Encrypted drive 'encrypted_daily:/data/': not deleting directories as there were IO errors
2021-01-27 11:07:12 INFO  : There was nothing to transfer
2021-01-27 11:07:12 ERROR : Attempt 2/3 failed with 1 errors and: couldn't list directory: googleapi: Error 401: Invalid Credentials, authError
2021-01-27 11:07:12 DEBUG : vzdump-qemu-100-2020_07_03-17_20_59.vma.gz: Excluded
2021-01-27 11:07:13 ERROR : : error reading destination directory: couldn't list directory: googleapi: Error 401: Invalid Credentials, authError
2021-01-27 11:07:13 DEBUG : Encrypted drive 'encrypted_daily:/data/': Waiting for checks to finish
2021-01-27 11:07:13 DEBUG : Encrypted drive 'encrypted_daily:/data/': Waiting for transfers to finish
2021-01-27 11:07:13 ERROR : Encrypted drive 'encrypted_daily:/data/': not deleting files as there were IO errors
2021-01-27 11:07:13 ERROR : Encrypted drive 'encrypted_daily:/data/': not deleting directories as there were IO errors
2021-01-27 11:07:13 INFO  : There was nothing to transfer
2021-01-27 11:07:13 ERROR : Attempt 3/3 failed with 1 errors and: couldn't list directory: googleapi: Error 401: Invalid Credentials, authError
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         2.3s
2021/01/27 11:07:13 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         2.3s

2021/01/27 11:07:13 DEBUG : 6 go routines active
2021/01/27 11:07:13 Failed to sync: couldn't list directory: googleapi: Error 401: Invalid Credentials, authError
root@server:~# 

Edited the existing remote using rclone config option e) Edit existing remote and refreshed the token for gdrive: . Let's see if the error will come back again.

If it helps with troubleshooting, I run the sync command once every 24 hours sometimes more often should the need arise.

Hmm, not sure what caused that. Google tokens always have an expiry time as far as I know, so I suspect there was some problem renewing the token.

That is what I would have done. Note there is a shortcut to doing that rclone config reconnect gdrive:

7 days later and the error has returned.

The following is in my log:

2021/02/04 06:13:10 Failed to sync: couldn't list directory: Get "https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%$
Response: {
  "error": "invalid_grant",
  "error_description": "Token has been expired or revoked."
}

But this time in the rclone config the token has a date and time(before running the rclone config reconnect mentioned below):

"expiry":"2021-02-03T08:00:58.970110435-07:00"}

Ran the rclone config reconnect gdrive: and I was able to refresh the token.

Can the rclone config reconnect be automated on headless machines?

The only other thing I can think of is would creating a "service account" for my google api prevent the need for a token to be refreshed?

Hmm, you should never need to run rclone config reconnect gdrive: unless you don't use rclone for a long time.

Yes you can do that and it will work. It shouldn't be needed though!

I run the sync command every 24 hours. Would that be long enough? But then why is it working for roughly 7 days and then causing an issue?

Is there anything else I could check on my system to see if it's causing problems? Date and time are correct.

If I do go the service account route. It should work with a plain old @gmail.com drive account correct?

It should be

I don't know.

Is there something special / unusual about your drive account?

You can create a service account with an gmail drive. I have one. But you can't get that service account to read your actual drive files with --drive-impersonate. You can do this if you have a google enterprise account.

As I understand it, when you make a service account, you are effectively creating a new user with its own 15 GB limit by default.

You can see this user in your service account JSON.

So what you can do is share a directory (or the root) with that service account email which will look something like "xxx@xxx.iam.gserviceaccount.com"

Then you can do this to discover the id of the shared folder

$ rclone lsf -F pi --drive-shared-with-me drivesa:
test/;0BzSw5cyzKmgvWi1CMXBnckJHSEE

Then you can access it like this

rclone lsf --drive-root-folder-id 0BzSw5cyzKmgvWi1CMXBnckJHSEE drivesa:

Any files you upload will be owned by the service account which isn't idea.

It would be better to fix the expiring credentials!

Not that I know of. It's a myusername@gmail.com account with 2TB of "Google One" purchased storage.

I think I'll try and avoid the service account route for now. I might even try making my own api again from scratch before moving to a service account.

I've changed my logging level to DEBUG in the hope it'll give more of a clue as to what's going on if/when the issue comes back. Disk space for the logs is not an issue at this time, but I'll keep an eye on disk usage to be sure it doesn't fill up the drive.

I have an identical account with 100GB of storage and it has never done this in the last 5 years!

I wonder if there is some setting in the client_id when you made that?

Thanks :slight_smile:

So I did some further digging. This may have been all my fault.

My OAuth consent screen was still in testing under the Publishing status. I also found this

A Google Cloud Platform project with an OAuth consent screen configured for an external user type and a publishing status of "Testing" is issued a refresh token expiring in 7 days.

There is currently a limit of 50 refresh tokens per Google Account per OAuth 2.0 client ID. If the limit is reached, creating a new refresh token automatically invalidates the oldest refresh token without warning. This limit does not apply to service accounts.

I've now changed the Publishing status to In production. So we'll see if that makes a difference. If the token needs to be refreshed hopefully it's just one last time.

So the error came back but I'm hoping that after I refresh it this time (after switching my google api to production) that it won't come back. Fingers crossed.

This is what was shown in the debug log:

2021/02/12 06:19:59 DEBUG : gdrive: Loaded invalid token from config file - ignoring
2021/02/12 06:19:59 DEBUG : gdrive: Token refresh failed try 1/5: oauth2: cannot fetch token: 400 Bad Request
Response: {
  "error": "invalid_grant",
  "error_description": "Token has been expired or revoked."
}
2021/02/12 06:20:00 DEBUG : gdrive: Loaded invalid token from config file - ignoring
2021/02/12 06:20:00 DEBUG : gdrive: Token refresh failed try 2/5: oauth2: cannot fetch token: 400 Bad Request
Response: {
  "error": "invalid_grant",
  "error_description": "Token has been expired or revoked."
}
2021/02/12 06:20:01 DEBUG : gdrive: Loaded invalid token from config file - ignoring
2021/02/12 06:20:01 DEBUG : gdrive: Token refresh failed try 3/5: oauth2: cannot fetch token: 400 Bad Request
Response: {
  "error": "invalid_grant",
  "error_description": "Token has been expired or revoked."
}
2021/02/12 06:20:02 DEBUG : gdrive: Loaded invalid token from config file - ignoring
2021/02/12 06:20:02 DEBUG : gdrive: Token refresh failed try 4/5: oauth2: cannot fetch token: 400 Bad Request
Response: {
  "error": "invalid_grant",
  "error_description": "Token has been expired or revoked."
}
2021/02/12 06:20:03 DEBUG : gdrive: Loaded invalid token from config file - ignoring
2021/02/12 06:20:03 DEBUG : gdrive: Token refresh failed try 5/5: oauth2: cannot fetch token: 400 Bad Request
Response: {
  "error": "invalid_grant",
  "error_description": "Token has been expired or revoked."
}
2021/02/12 06:20:04 DEBUG : gdrive: Loaded invalid token from config file - ignoring
2021/02/12 06:20:04 DEBUG : gdrive: Token refresh failed try 1/5: oauth2: cannot fetch token: 400 Bad Request
Response: {
  "error": "invalid_grant",
  "error_description": "Token has been expired or revoked."
}
2021/02/12 06:20:05 DEBUG : gdrive: Loaded invalid token from config file - ignoring
2021/02/12 06:20:05 DEBUG : gdrive: Token refresh failed try 2/5: oauth2: cannot fetch token: 400 Bad Request
Response: {
  "error": "invalid_grant",
  "error_description": "Token has been expired or revoked."
}
2021/02/12 06:20:06 DEBUG : gdrive: Loaded invalid token from config file - ignoring
2021/02/12 06:20:06 DEBUG : gdrive: Token refresh failed try 3/5: oauth2: cannot fetch token: 400 Bad Request
Response: {
  "error": "invalid_grant",
  "error_description": "Token has been expired or revoked."
}
2021/02/12 06:20:07 DEBUG : gdrive: Loaded invalid token from config file - ignoring
2021/02/12 06:20:07 DEBUG : gdrive: Token refresh failed try 4/5: oauth2: cannot fetch token: 400 Bad Request
Response: {
  "error": "invalid_grant",
  "error_description": "Token has been expired or revoked."
}
2021/02/12 06:20:08 DEBUG : gdrive: Loaded invalid token from config file - ignoring
2021/02/12 06:20:08 DEBUG : gdrive: Token refresh failed try 5/5: oauth2: cannot fetch token: 400 Bad Request
Response: {
  "error": "invalid_grant",
  "error_description": "Token has been expired or revoked."
}
1 Like

:crossed_fingers: !

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.