I’ve uploaded files using rclone to a Google Drive. I can see correct contents on the drive using rclone ls but when viewing the drive with the web UI (Chrome) the contents aren’t visible. (Also, rclone ls reveals only what it uploaded, none of the other existing content on the drive.)
My goal is to be able to view the synced contents using the web UI.
Thanks in advance if someone can point me in the right direction.
Details:
The upload was performed using rclone copy
Authentication is done with a Service Account
Specifying a root_folder_id for a directory I made in the web UI caused a 404 when trying to upload to it.
It’s almost like the files are being uploaded to another drive (I checked all my other accounts to see if the files were going somewhere else. They aren’t.)
The Google account I’m using is part of a G Suite account.
Thanks for the fast response. Here’s what I’ve found so far…
Using the --drive-impersonate flag gets this response from Google Drive:
Response: {
"error" : "unauthorized_client",
"error_description" : "Client is unauthorized to retrieve access tokens using this method."
}
I tried giving the service account appropriate domain-wide authority to go into the drives of users in that domain, but kept getting the same error.
I’m working on this from a different angle now. I’m attempting to log in using regular Client ID + Client Secret credentials, no service account. I’m running into redirect uri mismatch problems right now.
I should mention that I’m setting up rclone on a remote/headless machine.
When setting up the remote in rclone config I enter the Client ID and Client Secret.
In my browser, I add http://127.0.0.1:53682 as an authorized redirect URI, as rlcone asked me to.
In rclone, I type n since I’m on a headless machine and rclone gives me a link to copy/paste into my browser. However, this link’s redirect_uri is of the urn:xxxx:xx:oauth:2.0:xxx variety. This produces a 400 error when I go to the link in my browser. (Google’s message: “The redirect URI in the request, [redacted], can only be used by a Client ID for native application. It is not allowed for the WEB client type.”)
Just to see what would happen, I replaced the value of the redirect_uri parameter with http%3A%2F%2F127.0.0.1%3A53682 (which us URL-encoded http://127.0.0.1:53682). Using this in the link takes me to Google Account login screen and allows me to authorize the app for access to Google Drive.
Upon clicking Allow, I get redirected to http://127.0.0.1:53682 with two parameters: state and code. I tried copy-pasting the code value into rclone but it then responds with “Failed to configure token: failed to get token: oauth2: cannot fetch token: 400 Bad Request” Response: { "error" : "redirect_uri_mismatch" }.
That’s where I am now. Maybe I’m going about this all wrong but this is the closest I’ve come to getting this working.
Success! I’ll document thoroughly what I did in this post for others in the future. I hope it saves someone else a lot of time.
Setting up a Service Account which can act on behalf of any user in a G Suite domain
This post explains how I was able to set up a service account within a G Suite domain which can use the --drive-impersonate flag to enter the Google Drive of any user on that domain. Four key takeaways:
Domain-wide Delegation must be enabled for the service account
Enabling DwD should create a Client ID for that service account
That Client ID must be authorized by the admin of the G Suite domain to access drives of users in that domain
All of this should be done using the account which administers the G Suite domain. (This may not be strictly necessary but for me it seems to have been crucial.)
Create a project if you haven’t already. Be sure that when you create the project, your domain is selected as the organization, not “No organization.” Be sure to enable the Google Drive API.
Go to Credentials and set up a new Service Account.
Create a key for this service account. This should automatically download a JSON file with the credentials.
Enable Domain-wide Delegation for this service account. (Click on the ‘edit’ item in that service account’s menu. You’ll get a dialog that lets you check the ‘Enable G Suite Domain-Wide Delegation’ checkbox.)
Enabling DwD creates a Service account client. When you look at all credentials, you will see “Client for [service account]” has been added. That client will have a Client ID. Copy it to your clipboard or leave this tab open.
G Suite Admin
Now we have to authorize the Client ID of the “Client for [service account]” on your G Suite domain. Go to https://admin.google.com (still logged in to the administrative account).
Click on Security > Advanced settings > Manage API client access
In the row for adding a new client, paste the Client ID of the service account client. In the API Scopes field, you can enter the appropriate scopes. For rclone it is most likely https://www.googleapis.com/auth/drive
rclone
Run rclone config. Set up your remote without entering a Client ID or Client Secret, just the JSON file with the credentials for the service account.
You should now be able to use rclone using the --drive-impersonate flag to act as if you were any user on that domain. For example:
rclone ls --drive-impersonate user@example.com remote:
Note
I tried the above previously, but using a service account which was created by a user other than the G Suite administrator. That service account was not allowed access, even when approved by the administrator. I don’t know if others will have this problem as well, maybe it’s not necessary to use a service account created by the admin, but in my case it definitely seemed to be what finally made it work.