Rclone missing error code when SAs have no permission

Suspected Bug
1

What is the problem you are having with rclone?

rclone size/lsd/etc does not return an error code or an error when trying to access with a service account that has no access. Instead it shows an empty directory with no error code/message.

Potentially very dangerous:
rclone sync deletes all files in the destination if you use a service account file that has permission in the destination but not in the source remote.

Has been posted in github

What is your rclone version (output from rclone version)

rclone v1.50.2-060-ga7d65bd5-beta
same behavior with 1.50.1 release

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Same behavior on
Ubuntu
Mac OS
Win 10

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

ls/lsd/lsf
size
sync

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

root@foo:/# rclone version
rclone v1.50.2-060-ga7d65bd5-beta

  • os/arch: linux/amd64
  • go version: go1.13.4

root@foo:/# rclone size source:
Total objects: 6
Total size: 11.144 MBytes (11685834 Bytes)

root@foo:/# rclone size source: --drive-service-account-file=/sa/1.json
Total objects: 6
Total size: 11.144 MBytes (11685834 Bytes)

root@foo:/# rclone size source: --drive-service-account-file=/sa/2.json
Total objects: 0
Total size: 0 Bytes (0 Bytes)

root@foo:/# rclone sync source: dest: --drive-service-account-file=/sa/1.json -vP
2019-11-25 16:28:55 INFO : Google drive root '': Waiting for checks to finish
2019-11-25 16:28:55 INFO : Google drive root '': Waiting for transfers to finish
2019-11-25 16:28:58 INFO : mtd.bin.docx: Copied (server side copy)
2019-11-25 16:28:58 INFO : test1/d.txt: Copied (server side copy)
2019-11-25 16:29:00 INFO : test1/e.txt: Copied (server side copy)
2019-11-25 16:29:00 INFO : test2/a.txt: Copied (server side copy)
2019-11-25 16:29:01 INFO : test2/b.txt: Copied (server side copy)
2019-11-25 16:29:01 INFO : test2/c.txt: Copied (server side copy)
2019-11-25 16:29:01 INFO : Waiting for deletions to finish
Transferred: 11.144M / 11.144 MBytes, 100%, 1.593 MBytes/s, ETA 0s
Transferred: 6 / 6, 100%
Elapsed time: 6.9s
2019/11/25 16:29:01 INFO :
Transferred: 11.144M / 11.144 MBytes, 100%, 1.593 MBytes/s, ETA 0s
Transferred: 6 / 6, 100%
Elapsed time: 6.9s

root@foo:/# rclone sync source: dest: --drive-service-account-file=/sa/2.json -vP
2019-11-25 16:29:09 INFO : Google drive root '': Waiting for checks to finish
2019-11-25 16:29:09 INFO : Google drive root '': Waiting for transfers to finish
2019-11-25 16:29:09 INFO : Waiting for deletions to finish
2019-11-25 16:29:10 INFO : test2/c.txt: Deleted
2019-11-25 16:29:10 INFO : mtd.bin.docx: Deleted
2019-11-25 16:29:10 INFO : test1/d.txt: Deleted
2019-11-25 16:29:10 INFO : test1/e.txt: Deleted
2019-11-25 16:29:10 INFO : test2/a.txt: Deleted
2019-11-25 16:29:11 INFO : test2/b.txt: Deleted
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks: 0 / 6, 100%
Deleted: 6
Elapsed time: 1.8s
2019/11/25 16:29:13 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks: 0 / 6, 100%
Deleted: 6
Elapsed time: 1.8s

2

Are you sure it isn't just showing the service account's space? Service accounts are in some ways equivalent to users (they have email addresses!) and they also get the 15GB free space. So if you aren't using --drive-impersonate that is what you will see.

3

This is with Team Drives, I believe.

In those cases, if it fails to auth for that teamdrive, what is the expected behavior ?

4

Hi @ncw ,

Exactly as Darth mentioned, these are Shared Drives/Team Drives.

The dangerous outcome is that if someone inadvertently uses a service account that is authed and works with a destination but not the source then it can wipe out their entire destination drive.

NOTE: Checked and found same behavior with My Drive.
This seems to be happening only with the root of the remote. If you specify a folder as source then you get the correct error.

root@foo:# rclone lsd source:test --drive-service-account-file=/sa/1.json;echo $?
2019/11/25 20:42:27 ERROR : : error listing: directory not found
2019/11/25 20:42:27 Failed to lsd with 2 errors: last error was: directory not found
3
root@foo:# rclone lsd source: --drive-service-account-file=/sa/1.json;echo $?
0
root@foo:# rclone lsd source:/ --drive-service-account-file=/sa/1.json;echo $?
0
root@foo:#```
5

Can you do a log with -vv --dump bodies so I can see exactly what drive returns? Feel free to edit for privacy!

6 
rclone lsd source: --drive-service-account-file=/sa/1.json -vv --dump bodies
2019/11/25 23:28:58 DEBUG : rclone: Version "v1.50.2-060-ga7d65bd5-beta" starting with parameters ["rclone" "lsd" "source:" "--drive-service-account-file=/sa/1.json" "-vv" "--dump" "bodies"]
2019/11/25 23:28:58 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2019/11/25 23:28:58 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2019/11/25 23:28:58 DEBUG : HTTP REQUEST (req 0xc00053ba00)
2019/11/25 23:28:58 DEBUG : POST /token HTTP/1.1
Host: oauth2.googleapis.com
User-Agent: rclone/v1.50.2-060-ga7d65bd5-beta
Content-Length: 759
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip

assertion=eyJhbG123Ubw&grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer
2019/11/25 23:28:58 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2019/11/25 23:28:58 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2019/11/25 23:28:58 DEBUG : HTTP RESPONSE (req 0xc00053ba00)
2019/11/25 23:28:58 DEBUG : HTTP/1.1 200 OK
Transfer-Encoding: chunked
Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000
Cache-Control: private
Content-Type: application/json; charset=utf-8
Date: Mon, 25 Nov 2019 15:29:01 GMT
Server: scaffolding on HTTPServer2
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

ce
{
  "access_token": "ya2.c.Kl2123WvZw",
  "expires_in": 3600,
  "token_type": "Bearer"
}
0

2019/11/25 23:28:58 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2019/11/25 23:28:58 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2019/11/25 23:28:58 DEBUG : HTTP REQUEST (req 0xc0005fc200)
2019/11/25 23:28:58 DEBUG : GET /drive/v3/files?alt=json&corpora=drive&driveId=0AJ123PVA&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%29%2CnextPageToken&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%27123PVA%27+in+parents%29&supportsAllDrives=true HTTP/1.1
Host: www.googleapis.com
User-Agent: rclone/v1.50.2-060-ga7d65bd5-beta
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.13.0 gdcl/20191026
Accept-Encoding: gzip

2019/11/25 23:28:58 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2019/11/25 23:28:58 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2019/11/25 23:28:58 DEBUG : HTTP RESPONSE (req 0xc0005fc200)
2019/11/25 23:28:58 DEBUG : HTTP/1.1 200 OK
Transfer-Encoding: chunked
Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Mon, 25 Nov 2019 15:29:01 GMT
Expires: Mon, 25 Nov 2019 15:29:01 GMT
Server: GSE
Vary: Origin
Vary: X-Origin
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block

c
{"files":[]}
0

2019/11/25 23:28:58 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2019/11/25 23:28:58 DEBUG : 7 go routines active
2019/11/25 23:28:58 DEBUG : rclone: Version "v1.50.2-060-ga7d65bd5-beta" finishing with parameters ["rclone" "lsd" "source:" "--drive-service-account-file=/sa/1.json" "-vv" "--dump" "bodies"]
7

As NCW mentioned - because you do not see any errors but just a blank space, this sounds a whole lot like you are just viewing the SAs own default area (which will be empty, or maybe just have a "getting started" document. That can of course be confusing as it's probably not what you intended to use the SA for...

You can effectively think of an SA like a normal Google account in most ways. They have their own email address, their own drive-space ect. So the default space that will be shown is their own "personal" space - just like on any other account.

If you want to use an SA to access files located somewhere else (let's say on a shared drive) then you must make sure to tell rclone this in the same way that you would with any account, with the team-drive ID or the root-folder ID flags (which amounts to much the same thing effectively). This just let's rclone know which if the storage-spaces we will be working within. Unlike folders - spaces can't be navigated between directly in rclone and need to be specified by flag at the onset. Same applies if you wanted to work within "shared with me" or "my computers" spaces. Of course the SA needs to have access to whatever area that you want it to work with (which is usually not within it's own spaces).

So I'm not saying this is necessarily the case here, but it sure sounds an awful lot like it... and I would first of all eliminate this possibility - because this may not be a bug at all but a simple misunderstanding about how SAs work.

Feel free to ask for clarifications as needed.

8

The problem here is that the SA is being used to access a team drive which it doesn't have permissions for. In such a case, instead of exiting with a permissions error, its instead returning an empty list which is being treated as a valid response and causing the deletion of all files on the destination.

9

That may be something I have not specifically tested.
If you've verified that this is an issue then I trust you to know your stuff on this, but thanks for the clarification. Let me know if you want to to try to replicate - but it sounds you are already pretty sure about what the problem is...

In such a case I agree we should be exiting with an error. An accidental sync if a permission changed (which could potentially be outside of the users direct control) could be quite destructive.

10

Hi Stigma.

It is quite easy to replicate the behavior on Google Shared Drives. This could be extremely dangerous/unfortunate as rclone will delete the entire drive (accidental rclone purge , oops!).

  1. Config a remote Shared Drive source: with normal id/secret auth and no service accounts.

  2. Config Shared Drive dest: using either normal auth or a service account ( e.g. service_account_file = /sa/1.json )

  3. Ensure the email for the 1.json service account is a member in the dest Shared Drive but not the source Shared Drive.

  4. Add a few test files to both source and dest.

  5. Run `rclone sync source: dest: --drive-service-account-file=/sa/1.json

All the files are deleted.

Fortunately this has never happened to me. But one of our users noted the behavior and caught it before deleting all his files (smart boy, dry run to the rescue).

11

This is the cause of the problem - google did not return an error it just returned an empty listing and OK.

If google had returned an error then rclone would have reported it.

So I think this is a Google bug rather than an rclone bug...

I can't think of a sensible work-around - can you?

It might be worth looking in the google issue tracker to see if you can see a relevant issue - I had a look but couldn't see one.

Can you try uploading a file to this mysteriously empty directory? See if you get an error then?

12

I got some time today to debug this further and the cause looks to be the query parameter where we request the files only if the team drive id is in parents i.e. 123PVA+in+parents

If I send the request without this in the query, it returns the expected 403.

This is probably a bug from Google but can rclone do something to work around this? Perhaps an additional request without this parameter to check if we have the necessary permissions?

13

Interesting...

So I guess google is saying that since you don't have access to that parent then there are no files available or something.

A request without the parent returns all the files in the drive.

Does rclone about give an error?

14

You can't upload to the Shared Drive without auth, of course :wink:

Running a touch rclone touch source:test.bin --drive-service-account-file=/sa/1.json yields the normal error

2019/11/26 18:05:59 Failed to touch: googleapi: Error 404: File not found: 012345678PVA., notFound

If you omit --dsaf then touch/delete work fine.

There are a couple of workarounds depending on the use-case:

  1. Where I can read/write to the remote then I use rclone touch and rclone delete with a test.bin. This behaves the way we expect, giving correct error codes.

  2. If you cannot write/delete to the remote then an 'ugly' workaround is to test equivalency with and without --dsaf:

    i=rclone lsd remote: --drive-service-account-file=1.json
    j=rclone lsd remote
    if [ "$i" == "$j" ] ... etc.```

This works if the remote is not actually empty. If you want to be extra safe then you can run with rclone size rather than lsd. That is much slower for large remotes but if the results are not equal or empty then you bypass the rclone sync/copy.

15

No, rclone about doesn't throw an error. In fact, it doesn't even make any request for shared drives.

Yeah, I get that but ideally, this would only be done once to check the permissions. The following requests can be done normally with the filtered parent set.

16

Played around a bit more with different commands and found one which yields different errors for remotes with service account auth and those without service account auth (when using --drive-service-account-file in the command):

remote1 has id/secret auth and the team drive is not shared with the service account. This yields a 404 error
remote2 has service_account_file auth only and is shared with the service account. This yields a 403 error

rclone link remote1: --drive-service-account-file=/sa/1.json
2019/11/26 19:26:43 Failed to link: googleapi: Error 404: File not found: 0A12345678VA., notFound

rclone link remote2: --drive-service-account-file=/sa/1.json
2019/11/26 19:26:49 Failed to link: googleapi: Error 403: Cannot share top level folders of shared drives to domains or anyone., cannotShareTeamDriveTopFolderWithAnyoneOrDomains```
17

It would be easy enough to

when listing the root

  • if nothing is returned
  • list again with a limit of 1 file and without the parents and report error if any

@darthShadow do you want to have a go at a PR for this?

18

Honestly depends on whether you want it done within the next 2 months or not... :sweat_smile:

If nobody has picked it up by then, I can send a PR to add the checks.

19

I updated #3763 with the findings from this thread!

20

I've attempted to fix this here

https://beta.rclone.org/branch/v1.50.2-071-g11b5ba92-fix-sharedrive-beta/ (uploaded in 15-30 mins)

This change:
- calls teamdrives get on rclone about
- calls teamdrives get on a listing of the root which returned no entries

These will both detect a team drive which has the incorrect auth and
workaround the issue.