GCE Token Auto Refresh?

Help and Support
1

Thank you for the fabulous rclone! Without this life in the cloudverse would be so boring, and slow. Question:

I have a few instances set up on Google Compute Engine (GCE) w Ubuntu 16.04. Rclone is configured for several Google Team Drives and works perfectly most of the time. If I leave a GCE instance running the tokens refresh automatically and all runs well.

However, if I shut down an instance, sometimes for a day and sometimes for a week, and then restart I get odd behavior. Using the same config file with multiple Team Drives, some TDs get refreshed/current tokens and are usable while others have expired tokens which do not refresh. All TDs are in a single config file and belong to the same G Suite account.

Note that this is relatively new behavior. I have used the same GCE instances for almost a year, shutting some down and restarting them in the same way. The “expired token not being refreshed” issue only started a couple of months ago. Several of the team that I work with have experienced exactly the same behavior.

I find this problem on both old GCE instances and fresh ones that I created last week.

I can manually fix the problem by running rclone config edit teamdrive1 and clicking through until refresh token. Grab the new token then all works well. But I’d prefer not to do this if possible.

I’m also puzzled as to why some tokens expire while others refresh. There are 10 Team Drives configured in the same config file for the same Google account. Some days I restart the instance and all 10 work fine, while on other days 2 or 3 of then have expired tokens. Normally I would have expected either all of the tokens are refreshed or none of them are.

FYI, below is an extract from rclone config dump showing one working and one expired Team Drive (client_id, secret, tokens etc have all been sanitized, but appear to be fine in the actual dump). If I manually edit and refresh the token then the token and expiry are updated as you would expect.

“teamdrive1”: {
“client_id”: “123456789-abcdefgh.apps.googleusercontent.com”,
“client_secret”: “ABCDEFGHIJKL”,
“scope”: “drive”,
“team_drive”: “ABC123DEF456”,
“token”: “{“access_token”:“token12345678”,“token_type”:“Bearer”,“refresh_token”:“abcdefghijkl”,“expiry”:“2018-12-05T13:15:54.700947294Z”}”,
“type”: “drive”
},
“teamdrive2”: {
“client_id”: “987654321”,
“client_secret”: “lkjhgfdsa”,
“scope”: “drive”,
“team_drive”: “987IUY543VCX”,
“token”: “{“access_token”:“token1234567890”,“token_type”:“Bearer”,“refresh_token”:“abcdefghijklMNBV”,“expiry”:“2018-12-21T03:27:11.777643151Z”}”,
“type”: “drive”

SEPARATE QUESTION: Is there a single command that would show status (e.g. live/expired) for all configured drives? rclone config dump shows me what I need to see, but wondering if there is something more succinct.

2

:smile:

Did the original refresh token get copied around to lots of places? I wonder whether this might be a security measure by google if it sees a refresh token used by lots of different IPs then it disables it?

Some companies do this or similar - for instance box.com only allow their refresh tokens to be used once which means that you can’t copy config files about the place.

One solution might be to setup a service account - that should work and should never expire.

3

@ncw thank you for the quick reply.

The original refresh token in this case has only been used in the one GCE instance. Still puzzled, especially by the variant behavior of different TDs that were authorized at the same time.

The service account suggestion is a good one, thank you. I’ve used those in the past with GCEs and indeed never had an issue with them. A little more work on the front end but saves effort on the back end.

4

Yes there is something going on… I suspect some sort of security measure, but I don’t know for sure!

That is the recommended way of running unattended services so should work!

Though I’ve never had any problem with expiring refresh tokens with Google services :man_shrugging: