Google Cloud Permissions Error

Help and Support
#1

What is the problem you are having with rclone?

I am trying to copy files to a Google Cloud bucket and getting a permissions error. i.e. Failed to copy: googleapi: Error 403: user@example.com does not have storage.objects.delete access to...

I'm not sure why I need delete permissions for a copy command. It's my intention to only assign write and read/list commands to prevent an attacker from deleting backups.

What is your rclone version (output from rclone version)

rclone v1.54.0

  • os/arch: linux/amd64
  • go version: go1.15.7

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Ubuntu 20.04 with 64 bit 5.4.78-2-pve kernel, running in a Linux container.

Which cloud storage system are you using? (eg Google Drive)

Google Cloud

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Paste command here

rclone copy /mnt/cameras CryptCameras:lalala

The rclone config contents with secrets removed.

[GoogleCloud]
type = google cloud storage
storage_class = COLDLINE
token = {"access_token":","token_type":"Bearer","refresh_token":"","expiry":"2021-02-20T14:48:57.699451649-06:00"}
project_number = 
object_acl = private
bucket_acl = private
location = us

[CryptCameras]
type = crypt
remote = GoogleCloud:lalala
filename_encryption = standard
directory_name_encryption = true
password = 
password2 = ```

#### A log from the command with the `-vv` flag  
<!-- You should use 3 backticks to begin and end your paste to make it readable.  Or use a service such as https://pastebin.com or https://gist.github.com/   -->

https://pastebin.com/mypHwmq7
#2

It isn't immediately obvious to me either...

I had a look through your log for the failing file and I see

     18:2021/02/20 14:42:27 DEBUG : driveway/datadir0/hiv00001.mp4: Modification times differ by -79h40m23.291943188s: 2021-02-16 23:51:07.097333463 -0600 CST, 2021-02-13 16:10:43.805390275 -0600 CST
    323: *                driveway/datadir0/hiv00001.mp4: 12% /256M, 1.061M/s, 3m31s
    334: *                driveway/datadir0/hiv00001.mp4: 18% /256M, 595.479k/s, 5m57s
    345: *                driveway/datadir0/hiv00001.mp4: 25% /256M, 203.626k/s, 16m5s
    356: *                driveway/datadir0/hiv00001.mp4: 31% /256M, 60.343k/s, 49m46s
    367: *                driveway/datadir0/hiv00001.mp4: 37% /256M, 50.565k/s, 54m0s
    378: *                driveway/datadir0/hiv00001.mp4: 43% /256M, 34.531k/s, 1h11m10s
    391: *                driveway/datadir0/hiv00001.mp4: 50% /256M, 191.947k/s, 11m22s
    402: *                driveway/datadir0/hiv00001.mp4: 56% /256M, 125.713k/s, 15m12s
    413: *                driveway/datadir0/hiv00001.mp4: 68% /256M, 355.196k/s, 3m50s
    424: *                driveway/datadir0/hiv00001.mp4: 75% /256M, 75.482k/s, 14m28s
    435: *                driveway/datadir0/hiv00001.mp4: 81% /256M, 159.136k/s, 5m8s
    446: *                driveway/datadir0/hiv00001.mp4: 93% /256M, 456.261k/s, 35s
    450:2021/02/20 14:55:25 ERROR : driveway/datadir0/hiv00001.mp4: Failed to copy: googleapi: Error 403: user@example.com does not have storage.objects.delete access to 2nynoj984pyy79bb..., forbidden

So it looks like rclone tried to delete the file at the end of hte transfer. This can happen if the transfer was corrupt, but there should have been a log Removing failed copy if that happened.

Can you do your command with -vv --dump headers and post that output please?

#3

Thank you for your help, Nick.

I've posted the requested logs here. Since I'm not sure if it will expose authentication information, I've password-protected the post. I'll send it to you in a DM.

#4

Here is the request and the response

2021/02/24 10:29:04 DEBUG : HTTP REQUEST (req 0xc000492100)
2021/02/24 10:29:04 DEBUG : POST /upload/storage/v1/b/2nynoj984pyy79bb/o?alt=json&name=noi3lv762damf4mpdu3icjrnkc%2F7ng18l9ik1j4er938jimasifas%2Ft8b0m31mbo2nrd1k0ink6pirss&predefinedAcl=private&prettyPrint=false&uploadType=resumable&upload_id=ABg5-Uzq_LSzlACJg6HzBSp9AKVJ2edme21seqLflkraBzXXvdm1ymMoUXXronO3MUZKjW16zne7pceZnZowclx4hfQ HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.54.0
Content-Length: 65568
Authorization: XXXX
Content-Range: bytes 268435456-268501023/268501024
Content-Type: 
X-Guploader-No-308: yes
Accept-Encoding: gzip

And the response

2021/02/24 10:29:04 DEBUG : HTTP RESPONSE (req 0xc000492100)
2021/02/24 10:29:04 DEBUG : HTTP/2.0 403 Forbidden
Content-Length: 432
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Wed, 24 Feb 2021 16:29:04 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ABg5-Uzq_LSzlACJg6HzBSp9AKVJ2edme21seqLflkraBzXXvdm1ymMoUXXronO3MUZKjW16zne7pceZnZowclx4hfQ

Which caused the error

2021/02/24 10:29:04 ERROR : driveway/datadir0/hiv00002.mp4: Failed to copy: googleapi: Error 403: nothing@example.com does not have storage.objects.delete access to 2nynoj984pyy79bb/noi3lv762damf4mpdu3icjrnkc/7ng18l9ik1j4er938jimasifas/t8b0m31mbo2nrd1k0ink6pirss., forbidden

That is strange because this is the second part of a multipart upload, and the first part went fine

The only thing I can think of is that is overwriting an existing file - could that be the case?

#5

Yes, that file does exist. Does this happen because the initial transfer was interrupted? Would this additional permission be needed for this scenario?

#6

OK

Possibly or was there a file with that name already?

It looks like it, yes.

#7

I was able to successfully copy my data with the permission added. Thanks for your help.

Perhaps a feature can be added to avoid this. Maybe it would append a -2 or something to the end of the file with a flag like --no-del. Just throwing some ideas out here.

1 Like
#8

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.