Error message: 2021/02/25 04:30:07 Failed to create file system for "G-Drive:": couldn't find root directory ID: Get "here... ... googleapis.com/drive/v3/files/root?alt=json&fields=id&prettyPrint=false&supportsAllDrives=true": couldn't fetch token - maybe it has expired? - refresh with "rclone config reconnect G-Drive:": oauth2: cannot fetch token: 400 Bad Request
Response: {
"error": "invalid_grant",
"error_description": "Token has been expired or revoked."
}
I am using rclone to backup from headless Linux to Google Drive. I have set up my own credentials (as I read that it runs faster) and have my own client id and secret. The backup all works fine for 1 week.
My problem is that the token from Google expires after 1 week and I have to intervene manually to re-authorise the token by running "rclone config reconnect :". This is less than ideal.
Reading this post: ...forum.rclone.org/t/google-drive-cannot-refresh-token-after-a-few-days/22036/18 (add https), I realise that this is a Google policy for apps in "Testing".
So my questions are...
Has anyone had any success getting Google tokens authorised permanently? Or at least longer than 1 week every time. I have looked into getting my OAuth connection verified, but it appears I need to own the website rclone.org to do this.
Can a headless "rclone config reconnect..." be scripted?
Would I be better to use the default client id? I presume it doesn't expire, but what is the performance impact?
The "project" was in testing, when I wrote this.
I believe that I have to have it verified to get it into production and I need to own the website to verify it (which clearly I can't do). That said, I am currently trying to verify it, so it is now sitting in production. That may be enough for it to sit in production permanently, or google may fail me and put it back into testing. Not sure.
Any ideas?
Have you got yours to successfully go through the verification process? Or did you start the process and never finish it (and it stayed in production anyway)?
To get through verification it says I need to provide (i) a home page which I own (I have one, but obvs not rclone.org), (ii) a privacy policy, (iii) a YouTube video describing how I'm going to use all the restricted scopes. This seems a tall order.
Thanks. I'll leave it as it is currently (in production, but not verified) and will report back on when the token next expires... Hopefully more than 1 week. Let's see.
Reply below from Google for my request for verification. I will try this...
Hi,
Thank you for your patience while we reviewed your project.
It looks like your app is only used by the people in your domain, so your project doesn’t need to be verified.
(Learn more about [internal vs. public users](link support .google .com/cloud/answer/6158849#public-and-internal)).
Note: internal use and personal use are different.
Applications for Internal Use
If this is correct, please let us know by replying to this email. We'll then close your request, and you can update your project from public to internal by following these steps:
Sign-in to[ Google Cloud Console](link console. cloud .google .com)
Select the project ID: RClone (id: <>)
Go to OAuth Consent Screen under APIs & Services
Go to User Type
Select Make Internal
Click Save
You also need to associate your project with your organization by following the steps below:
Create an Organization by following the [Quickstart Using Organizations]( link cloud .google .com/resource-manager/docs/quickstart-organizations) instructions.
Migrate the project into the organization you created as shown in [Migrating Existing Projects into the Organization](link cloud .google .com/resource-manager/docs/migrating-projects-billing).
Users in your organization can now use the app to directly access [OAuth scopes](link developers .google .com/identity/protocols/googlescopes) without any verification steps.
Note: Internal use and personal use are different. Please refer to "[What app types are not applicable for verification?](link support .google .com/cloud/answer/9110914?hl=en#skip)".
Applications for Personal Use
If your application is for personal use (fewer than 100 total users), your project does not require verification. It is recommended that you change your projects publish state from production to testing and continue to use your app under testing mode.
None of the Above
If your app is going to be used by people outside of your domain, reply to this email so that we can proceed with the current verification process.
Thanks. I'll leave it as it is currently (in production, but not verified) and will report back on when the token next expires... Hopefully more than 1 week. Let's see.
is that it did not work. The token expired after 1 week as before.
I will now try setting the project to internal as specified above and see how that goes...
Unfortunately, you can only set your project to internal if you are a Google Workspace user (i.e. you have a paid-for subscription to the Google Suite), which I am not. So I have gone back to Google, pointed this out, asked for further suggestions, or suggested that we proceed my project through verification...
I'm also running into this reoccurring weekly 'reset' and also after following all the steps of the gdrive doc. Should non GSuite users use the shared API? Looking forward to updated documentation.
Because I could not make my app internal, as I am not a Google Workspace user, I went back to Google with this:
Thank you for your response. You are right that this is an internal use of the API, but because I am not a Google Workspace user the option to make the project Internal is greyed out, so it appears this will not work for me.
Because of this, please can I proceed with getting the application verified as this seems to be the only option to stop the authorisation expiring every week (unless you have another idea)?
To which they replied...
Dear Developer,
Thank you for reaching out.
For further guidance on how to mark your application as internal-only so that verification is no longer required, please refer to the following [section] of our [FAQ]
You may also refer to [General Support] for all your support options.
The links provided did not work, but checking on Google support website it still says:
Internal apps: if your app is an internal web app for users in the same G Suite domain and the app is associated with a Cloud Organization that all of your users belong to, you don't need to go through verification. For more information, see [public and internal applications].
So I think I cannot set up this app as internal as you have to be a workspaces customer.
However there is 1 further update... since I sent that email on 5 Mar 2021, my OAuth token has not expired - it's been 3 weeks. So I don't know if Google did something for me (without telling me) or some policy has changed, but my problem appears to have gone away - at least for the moment.
Nick
I'm having the same problem and the way I see it, it is thus impossible to use Rclone with Google Drive configured as a scheduled task because of this. I have it set up on a remote server, so performing interactive re-authorization in a web browser is impossible.
Sadly using a so-called service-account is not an option for me either, because service-accounts can only read/write shared files but not delete them.
I understand this is not Rclone's fault at all, it is just unbelievable how user-unfriendly and frustrating the whole Google Drive experience is. I guess the best solution is to move to a different cloud storage provider.
Note, Hans_Blix that you always have the option of using rclone's default client id, which I assume will never expire.
Also I don't know what you mean by a service-account. I have a normal Google account to create my own client id and I am able to read/write/delete my files. If you cannot delete shared files, that may be a standard GDrive permission as you are not the file owner.
And finally, it is now over a month since my client id last expired (5 March 2021) and it is still going strong, so my problem seems to have gone away.
So it appears that Google just isn't clear about things. From one of the pages they linked you (snipped just for relevance)
What app types are not applicable for verification?
You do not need to submit your app for review if it's going to be used in any of the following scenarios:
Personal Use: The app is not shared with anyone else or will be used by fewer than 100 users. Hence, you can continue using the app by bypassing the unverified app warning during sign-in.
SMTP/IMAP/WP: The app is used to send emails through WordPress, or similar single account SMTP plug-ins.
Development/Testing/Staging: If your app is in development/testing/staging mode and not ready to be publicly accessible, then you do not need to submit your app for verification. Note that your app will be subject to the unverified app screen and the 100-user cap will be in effect when an app is in development/testing/staging.
Development/Testing/Staging: If your app’s publishing status is set to “Testing” and not “In production”, then you do not need to submit your app for verification. Note that your app will be subject to the unverified app screen and the 100-user cap will be in effect when an app is in development/testing/staging. Learn more about Publishing status.
I.e., it appears as though for production applications that are used only by you (/less than 100 people), it's fine, because they make a distinction between "Testing" "Production", and "Personal" use for "External" apps. Testing and Personal use don't need verification, but you'll get that warning. Testing also has that 7 day limitation.
While this may be irrelevant to the use case, it brings up an interesting question: does Google actually revoke app credentials for unverified apps they deem to not be personal, and if so, what causes them to deem an application not personal? 101 users? Because if so that's a silly metric.
Edit for clarification: ex, I just hit "publish app". Now, I have a maximum of 100 users (as listed in the API console), and it says I "need" to prepare/submit for verification because of the scopes involved, but this appears to only be to remove the 100 user restriction, as well as to remove a newly added "scary" safety screen, similar to Chrome's bad ssl "advanced+proceed/back to safety" page..