I can't create a unit with Service account

Hi guys.
I don't know what I'm doing wrong and maybe you can guide me.

What is the problem you are having with rclone?

I can't create a unit with Service account

What is your rclone version (output from rclone version)

rclone v1.51.0

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Debian 9, 64bits

Which cloud storage system are you using? (eg Google Drive)

Google Drive Gsuite

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone -vvv --drive-impersonate user@domain --drive-service-account-file sa-gen/accounts/11.json lsd probando:

The rclone config contents with secrets removed.

[probando]
type = drive
scope = drive
service_account_file = /home/user/sa-gen/accounts/11.json
team_drive = ************** (added by hand as it does not list the units)

I tried creating it with my client_id and my secret and leaving those fields blank and the problem persists

A log from the command with the -vv flag

2020/08/23 23:53:15 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "-vvv" "--drive-impersonate" "user@domain" "--drive-service-account-file" "sa-gen/accounts/11.json" "lsd" "probando:"]
2020/08/23 23:53:15 DEBUG : Using config file from "/home/user/.config/rclone/rclone.conf"
2020/08/23 23:53:16 ERROR : : error listing: couldn't list directory: Get https://www.googleapis.com/drive/v3/files?alt=json&corpora=drive&driveId=**************&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%27******************%27+in+parents%29&supportsAllDrives=true: oauth2: cannot fetch token: 401 Unauthorized
Response: {
  "error": "unauthorized_client",
  "error_description": "Client is unauthorized to retrieve access tokens using this method, or client not authorized for any of the scopes requested."
}
2020/08/23 23:53:16 Failed to lsd with 2 errors: last error was: couldn't list directory: Get https://www.googleapis.com/drive/v3/files?alt=json&corpora=drive&driveId=*****************&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%27*****************%27+in+parents%29&supportsAllDrives=true: oauth2: cannot fetch token: 401 Unauthorized
Response: {
  "error": "unauthorized_client",
  "error_description": "Client is unauthorized to retrieve access tokens using this method, or client not authorized for any of the scopes requested."
}

I explain what I want to do.
I have a 15tb server that I have to deliver within 5 days and I would like to upload all this to google.
I created a shared drive and set the group user@domain as administrator and created a couple of user accounts to get around the 750gb/day limit
I followed the steps of https://rclone.org/drive/#service-account-support to create the accounts and Delegating domain-wide authority to the service account and the disk has the user@domain group as administrator
The issue is that when adding the drive to rclone and choosing it as team drive, it does not recognize the drives

y) Yes
n) No (default)
y/n> y
Fetching team drive list...
Listing team drives failed: googleapi: Error 404: File not found: ., notFound
Choose a number from below, or type in your own value
Enter a Team Drive ID>

If I add it there by hand, it throws me the following when I want to list it

2020/08/23 23:32:51 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "-vvv" "--drive-impersonate" "user@domain" "lsd" "probando:"]
2020/08/23 23:32:51 DEBUG : Using config file from "/home/user/.config/rclone/rclone.conf"
2020/08/23 23:32:51 ERROR : : error listing: failed to get Team/Shared Drive info: googleapi: Error 404: Shared drive not found: ***************, notFound
2020/08/23 23:32:51 Failed to lsd with 2 errors: last error was: failed to get Team/Shared Drive info: googleapi: Error 404: Shared drive not found: ******************, notFound

What am I doing wrong?
Thanks!

EDIT: Format.

hello and welcome to the forum,

not sure what the exact problem is.

i would suggest to update rclone to latest stable v1.52.3 and test again.
https://rclone.org/install/#script-installation

There's a few more steps you need to take. You need to take both of these steps for it to work.

Did you share your team drive with the your service account user?

You need to share the team drive with the service account by adding the service account email address as a content manager. Uncheck the Notify members box. Once you do that, it will work.
You can get the service account email from the json file, look for client_email. You can also get it by going to https://console.developers.google.com/apis/credentials

You also need to enable the Google Drive API from the console, make sure you're logged in as your gsuite user. Enable The Google Drive API. Should just be a button click.

Once you enable the drive API and share the service account with the drive, it should work.

The reason why the team drive prompt was blank is because you didn't give the service accout access to any team drives. You have to share the drive like you would any user.

You can get the drive id yourself. It's an alphanumeric ID. Go to the google drive website, browse to your team drive and then look at the browser address bar for the url. In the Url is the teamdrive ID.

for example:

https://drive.google.com/drive/folders/4CA2uEazExJCLOk6QYY

then in the config:
team_drive = 4CA2uEazExJCLOk6QYY

@asdffdsa @wavlinky
Thank you both for responding.
I did what the two of you recommended and it's working now.
I think it was more on the side of adding the specific email account to the team drive users than for the lack of update.
Which generates a new question for me...I have the group group@domain, and inside of that group the 100 accounts generated by sa-gen (sa1@server-group.iam.gserviceaccount.com, sa2@server-group.iam. gserviceaccount.com...sa100@server-grupo.iam.gserviceaccount.com).
I had understood that, when adding group@domain as editor or administrator of the team drive, all accounts within it would have access to it (obviously, I'm wrong).
This means that if, at some point, I need more accounts to upload a larger amount of tb/day, do I have to manually add each of the service accounts?

@asdffdsa @wavlinky

teamwork! :upside_down_face:

1 Like

Each service account needs access to the team drive.
What I do is use a service account in the config but in my upload script, I override with

        --drive-service-account-credentials ""
        --drive-service-account-file "$gsa" # $gsa is a /path/to/serviceaccount.json

This is not wrong. If the group has access to the teamdrive, then all the service accounts in that group have access to the team drive.

I learned something new today. That is useful!

So I'm doing something wrong, because I can't make it work that way.
I have the group@domain as administrator of the unit, within the group I have 100 service accounts and, at the time of executing the command rclone -vvv --drive-service-account-file sa-gen/accounts/2.json lsd drive: (2.json account is not added manually to team drive) I get the error

2020/08/26 15:29:27 DEBUG : rclone: Version "v1.52.3" starting with parameters ["rclone" "-vvv" "--drive-service-account-file" "sa-gen/accounts/2.json" "lsd" "drive:"]
2020/08/26 15:29:27 DEBUG : Using config file from "/home/user/.config/rclone/rclone.conf"
2020/08/26 15:29:28 ERROR : : error listing: failed to get Team/Shared Drive info: googleapi: Error 404: Shared drive not found: ***************, notFound
2020/08/26 15:29:28 DEBUG : 6 go routines active
2020/08/26 15:29:28 Failed to lsd with 2 errors: last error was: failed to get Team/Shared Drive info: googleapi: Error 404: Shared drive not found: ***********, notFound

and when I do it with the 1.json account (whose email is added to the team drive) it accesses the team drive without problems.
Now I have it working by adding the necessary accounts manually (I don't need that many after all), but I would like to know what I'm doing wrong so that it doesn't work the way it's supposed to work

I assume you mean the group is the manager of the team/shared drive? And the Drive API is enabled in the project of the service account?

If that is your actual command then you probably need to amend the path to the json file, including a leading slash /.

/opt/sa-gen/accounts/2.json

What Darth said regarding adding SAs to a group and then the group to the team drive is entirely correct. If it is not working then you likely have something else misconfigured.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.