Google Drive requires manual refresh every 7 days

Suspected Bug
1

What is the problem you are having with rclone?

I followed the Rclone docs and created my own Google Drive token, but I still have to reresh it manually every 7 days.

Run the command 'rclone version' and share the full output of the command.

rclone v1.57.0-DEV

  • os/version: fedora 35 (64 bit)
  • os/kernel: 5.15.16-200.fc35.x86_64 (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.16.11
  • go/linking: dynamic
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

```rclone -v --log-file=LOG mount GDrive:Media media```

The rclone config contents with secrets removed.

```[GDrive]
type = drive
client_id = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.apps.googleusercontent.com
client_secret = XXXXXXXXXXXXXXXXX
token = {"access_token":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX","token_type":"Bearer","refresh_token":"1//XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX","expiry":"2022-01-25T14:26:38.291406768Z"}
team_drive = 




#### A log from the command with the `-vv` flag  
<!-- You should use 3 backticks to begin and end your paste to make it readable.  Or use a service such as https://pastebin.com or https://gist.github.com/   -->
2

hello and welcome to the forum,

there are a lot of posts in the forum about that, for example,
https://forum.rclone.org/t/invalid-credentials-autherror/21924/10?u=asdffdsa

as per the debug log you posted
couldn't fetch token - maybe it has expired? - refresh with "rclone config reconnect GDrive
so run rclone config reconnect GDrive

and best to use the offical rclone, not some dev version from some unknown source.
https://rclone.org/downloads/#script-download-and-install

3

Thanks. I had actually read that forum post earlier, and took it to mean that the problem is now solved. Clearly it hasn't in my case, but from what you say am I to understand that the version I have is registered on Google as "testing" rather than "production"? This is the standard Rclone package from Fedora 35 so if that's true I'm going to report it as a Fedora bug.

In the meantime I'll install the official version and see what happens.

4

it is not a bug, just another custom compiled development version and should not be used.
for some reason, there have been several recent users of fedora using that dev version.

https://forum.rclone.org/t/why-arent-cache-rclone-files-being-removed/28948/5

5

I regard it as a packaging bug. It's distributed as a standard Fedora package from the base repo, not the Testing repo, but doesn't function correctly because of this issue.

6

so you have installed and tested the offical rclone and you are NOT getting any errors?

7

I have installed the official version. It will take a week before I know if the error crops up again.

9

That is correct. The problem is with your client_id and client_secret rather than rclone, whether packaged from Fedora or not.

10

The client_id and client_secret were generated by the rclone config dialogue from the Fedora version. After installing the official version I simply copied them to the new config file rather than generating new ones. Does this mean that I will still have the same issue? Conversely, if I generate new ones from the official version, could I copy them to the Fedora version and thus avoid the issue?

11

That would very extremely odd if they generated their own during the config.

Can you share a screenshot of that version and the config screens?

The client ID and secret are just says to access the API. You would always want to generate your own and not use anyone elses.

12

To clarify: I mean that I am re-using the credentials that I originally generated when I installed the Fedora version, that's all. This is in response to ncw's comment about the problem being with the credentials rather than the package. If that's the case then I don't see how changing the installed version will make a difference. This is probably all due to my own hazy notion of how all this works.

13

Ah ok, the wording through me off as you said they were generated by the rclone config dialogue and you meant you entered them in to the rclone config dialogue as they were not generated by the rclone config.

Odds are the package isn't the issue as Fedora is a well known distribution and the package maintainer probably does just recompile it, but who knows. That's why support only comes for the official versions as it's too hard to guess. Whatbox does some crazy compile on their own and changes things as we've seen some strange things.

Are you generating a client ID/secret from your personal Google account or a GSuite account? Is it a testing or production config?

14

It's a personal account.

I've had a reply from one of the Fedora maintainers who suggests a possible fix. You can see it here:

https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org/thread/E4F7GIHBAFRMTQTU6Q53QX5SEKDSBPV2/

I haven't tested this yet as I'm still waiting the requisite 7 days to see if the official version resolves the issue, but it might be a way to allow the main distros to offer a fully-functional Rclone in their repos. I for one would strongly prefer this as I try to use the Fedora package system for everything when possible.

15

I think there is some confusion.

The build had nothing to do with the 7 day.

It's an issue with Google and with personal accounts and a non production oAuth screen, you only get 7 days. No build can fix that.

In this case, that leaves without a supported version of rclone unless you want to work with the package maintainer for any support issues as it is an unknown state.

16

I agree that there's some confusion and it's probably mine. However if "no build can fix that", then why is the official build the answer to my problem? What is it about the official build that a distro build can't replicate?

17

Let's take a step back actually as I'm speaking a bit too much in generalizations that aren't right 100% of the time as I was generalizing a bit.

With any Google Drive type application that leverages the Drive API, you have to use the oAuth method to get access (unless you use a service account but you mentioned our scope here is a personal account).

So if you have used a personal account as you can only use the production oAuth screen, the following link kicks in:

Using OAuth 2.0 to Access Google APIs | Google Identity | Google Developers

A Google Cloud Platform project with an OAuth consent screen configured for an external user type and a publishing status of "Testing" is issued a refresh token expiring in 7 days.

So if your screen looks like:

image

And you've used your project to create a client ID and secret, you can only use that for 7 days and it will not refresh and it will expire.

I didn't say it would be an answer to your problem. It's like going back to the manufacturer of something with a 3rd party part and asking them to fix it. "Bleeding edge" type repos do a good job usually keeping packages up to date and probably have really great maintainers. I'm not knocking them or passing any judgement as the challenge is, we simply don't know what they have done to make the package or what could be changed so it's not a great use of time to troubleshoot an unknown version hence the answer to use the proper stable build since it is a known quantity.

Back to my previous statement, it isn't about what they can't replicate, it's about a known and consistent starting point. Rclone has a self update function that can be used and it's easy to maintain as it is a single binary.

I use my repos for quite the number of things that are out there but for rclone, caddy and very recently pipx (had an issue as the packaged version with Ubuntu 20.04 LTS was ancient).

Back to my other point, can a build fix this issue? In a way yes.

The stable rclone version has it's own client ID/secret with it so, in theory, the maintained could make their own API keys and package them/distribute them as part of the binary. It's possible but I'd find that very unlikely as something a maintainer would do so I wanted to be specific on that part as I was a bit too broad before saying a build can't fix it as it can, but I can't imagine it being feasible.

That being said, you have a few options.

You can publish your app as I've heard people say they normally don't verify it and it works (not sure on that):

image

You can remove your own client ID/secret and use rclones default one (bit overloaded but it is an option).

You can find a friend with a GSuite account and they can create one for you that doesn't expire.

Hopefully, that's a bit more clear as I was trying to be detailed.

18

Thanks. I think I understand that now (until the next time :-). I actually do have a G Suite account as well as my personal one, so maybe I'll just use that. Am I right in assuming this won't affect the same Rclone credentials on my personal account?

(I think what makes all this harder to grasp for the newcomer is the idea that the user is effectively creating their own Google service. That's seems very strange because it's completely different from the normal way you access Google Drive, but I guess it makes sense when you think about it.)

19

I don't have a GSuite account anymore but if memory serves me (which it sometimes does not), I've always used my own GSuite API keys to access my GSuite Drive by using just a internal API key.

You'd have to validate / test to make sure you can use a client ID/secret via internal and when you enter that, you'd login with your personal credentials to get access to your personal Drive.

That's the access pattern you are trying to get right? Access to your personal drive through rclone?

20

Exactly. However a close reading of the (very long) Rclone man page seems to say that unattended access, which I guess is what I mean, is only for people with a "Service Account" or are G Suite admins. I only have a G Suite user account so I'm still not clear if this will work.

21

Perhaps bad wording as this all 'unattended' access as it's a CLI program doing it all behind the scenes and nothing is interactive in my mind.

A service account is for a GSuite domain so you can use that to 'impersonate' someone else. Back in the day, it had other unintended benefits for the quota use, but that's all rolled back to the user now I believe. A service account also doesn't get oAuth'ed so if you grab someone's service account, you are in (another reason why I wouldn't use them for me anyway).

I never used a service account nor had the need and only ever used a user account.

If you want to access a personal drive, you can use a GSuite client ID/secret that you generate as you just need to make sure you use the right account when you login.

client ID/secret is more "API" quota access.
Your user name/password login is what drive you are logging into and connecting.

They can be the same domain or different as if you submit no client ID/password, you use the default rclone client ID/secret without knowing it.