Should I have a separate client ID if I'm using a Google Drive service account?

Help and Support
1

The documentation for rclone, for use with Google Drive, says that ideally one should create one's own client ID. Otherwise, rclone will use its own client ID, in which case you can suffer from global throttling of that shared ID.

I'm using rclone with a Google Drive "service account", which has its own client ID. But I'm not sure whether or not I should also be using a separate client ID to address the issue described in the rclone documentation. (I couldn't find anything in the rclone documentation about this issue when using a service account.)

My rclone config file looks like this:

[archive]
type = drive
client_id = 
client_secret = 
scope = drive
root_folder_id = 
service_account_file = /Users/me/.secrets/[REDACTED].json
team_drive = [REDACTED]

Notice that the "client_id" field above is empty, but there IS a client_id in the json file pointed to by the "service_account_file" field above.

Should I also generate a separate client ID to put in the rclone config file? Or should I copy the "client_id" from the service account file into this file? Or is everything copacetic the way that it currently is?

I've searched this forum for an answer to this question, and I see that others have asked it before. But, unfortunately, I haven't been able to find a clear, definitive answer yet.

P.S. This is being used with an enterprise account, copying lots of files up to a shared drive.

2

I think the client_id in the rclone config refers to rclone whereas the client_id in the service file refers to the service account (which google views as a user).

So I think you will have to make your own client_id in the rclone config.

3

the service account file has everything in it already.

[gdrivesa]
type = drive
scope = drive
service_account_file = C:\data\rclone\rr\other\gdrive.sa\test01-xxx420-6f8144bb217f.json

contents of C:\data\rclone\rr\other\gdrive.sa\test01-xxx420-6f8144bb217f.json

{
  "type": "service_account",
  "project_id": "test01-309420",
  "private_key_id": "xxx",
  "private_key": "-----BEGIN PRIVATE KEY-----xxx-----END PRIVATE KEY-----",
  "client_email": "test02@xxx.iam.gserviceaccount.com",
  "client_id": "xxx",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/xxx.iam.gserviceaccount.com"
}
4

Oh nooooo. Conflicting answers.....

5

well, the docs shows that the client id is not needed when using a service fie

rclone config

n/s/q> n         # New
name>gdrive      # Gdrive is an example name
Storage>         # Select the number shown for Google Drive
client_id>       # Can be left blank
client_secret>   # Can be left blank
scope>           # Select your scope, 1 for example
root_folder_id>  # Can be left blank
service_account_file> /home/foo/myJSONfile.json # This is where the JSON file goes!
y/n>             # Auto config, n
6

I tried a test - I corrupted the built in client_id in the rclone source and my service account still worked, so I guess @asdffdsa is correct.

I wonder where the ops get accounted for though to the service account directly? If so you should be able to see that in the google console.

7

I tried a test - I corrupted the built in client_id in the rclone source and my service account still worked, so I guess @asdffdsa is correct.

Thanks for taking the time to figure this out!!!

8

well, the docs shows that the client id is not needed when using a service fie

That's because you don't ever need a client_id. If you don't provide your own, rclone has its own client ID that it will use. The problem with this is that there is a global limit on the number of Google Drive requests that everyone in the entire world using rclone can run up against when using the built-in client ID.

I.e., my use of rclone can effect the performance of rclone that you see, if neither of use gets our own client ID.

Thankfully, Nick, who is the "primary author of rclone", just did an experiment for our benefit, and it seems that you are right. I.e., if you are using a service account and haven't provided another client ID, rclone's built-in client ID is not used.

9

you may be new to the forum, but you sure learn fast :wink:

fwiw, another way to know, might be to run a command using rclone lsd archive: --dump=auth
it would show which token is used.
if that token matches the token in the remote, then that should prove which client id is used.

in my case, note: both are slightly redacted

rclone lsd gdrive: --dump=auth 
NOTICE: Automatically setting -vv as --dump is enabled
DEBUG : rclone: Version "v1.57.0" starting with parameters ["C:\\data\\rclone\\rclone.exe" "lsd" "gdrive:" "--dump=auth"]
DEBUG : Creating backend with remote "gdrive:"
DEBUG : Using config file from "C:\\data\\rclone\\rclone.conf"
DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
DEBUG : HTTP REQUEST (req 0xc000216900)
DEBUG : GET /drive/v3/files/root?alt=json&fields=id&prettyPrint=false&supportsAllDrives=true HTTP/1.1
Host: www.googleapis.com
User-Agent: rclone/v1.57.0
Authorization: Bearer ya29.A0ARrdaM8Qqw6__G8eVN8MYYfUv8ICFeLT67hzuFpwqM2qF4lgd70Oie3aZIYNQEqtLtYh126nfvNIX-zFqezBVkNFCoZx_UABVksGnjBHEEWSep0paj3ptdNBmBTTUP5gFz1n2Oqz3soqZHWcHbAqk4IX8FC4mIU
X-Goog-Api-Client: gl-go/1.17.2 gdcl/20210812
Accept-Encoding: gzip

and here is the remote

[gdrive]
type = drive
client_id = xxx.apps.googleusercontent.com
client_secret = xxx
scope = drive
token = {"access_token":"ya29.A0ARrdaM8Qqw6__G8eVN8MYYfUv8ICFeLT67hzuFpwqM2qF4lgd70Oie3aZIYNQEqtLtYh126nfvNIX-zFqezBVkNFCoZx_UABVksGnjBHEEWSep0paj3ptdNBmBTTUP5gFz1n2Oqz3soqZHWcHbAqk4IX8FC4","token_type":"Bearer","refresh_token":"1//0h9C5MPJwUJo4CgYIARAAGBESNwF-L9Ir4NoazoXYt8B1fHhAegRcmjISmndFatlUt_SYstasLG_zDjILrRxoeql4UKhWSsaR02k","expiry":"2022-04-05T19:23:25.5227561-04:00"}
10

and not sure this is proof,
when running rclone lsd --drump=auth with a remote using a service account.
the token is different format.

token using remote with client id, not using service account, from example up above.
Authorization: Bearer ya29.xxx

token using remote using service account
Authorization: Bearer ya29.c.xxx

11

When you use a service account, just check the API console and you can see hits on the service key.

12

Ah ha! So service accounts are definitely their own clients.

We should probably put that in the docs about creating your own client id that it isn't necessary if you are using a service account?

13

For the unenlightened, does "service account" mean the same as "Google Workspace account"?

14

No.

You have a user account. A user account logs in with a user name and password. A user can use none, one or many client ID/passwords to use Google's API services.

A service account has no password, uses it's own embedded client ID/secret and allows for no authentication as that's all tied into the account file. Someone gets the file, they have access to whatever you provisioned it for. That's generally meant for secured/standalone server type environments when you don't want to use a user.

15

Thanks. I assume that means that service accounts are available via Google Cloud rather than Google Drive, as Drive only appears to offer user accounts.

16

I'm not sure what you mean. You can use Service Accounts with Google Drive.

It's documented here:

Google drive (rclone.org)

17

Clearly more reading is in order.

18

take a close look at my example from up above, using gdrive with a service account.

19

I don't know what that means. Are you asking me to read something else or saying you need to read something? I'm happy to help clarify but I'm just not sure what you are trying to say.

20

I'm referring to myself. I find a lot of the terminology around Google Drive pretty obscure so I need to become more familiar with it. Also the Rclone docs seem to assume a certain familiarity with this terminology and aren't really oriented towards the casual user.