WebDAV requires delete to edit : GCS backend

awoodobvio · September 24, 2021, 6:31pm

What is the problem you are having with rclone?

I'm using "rclone serve webdav" with an alias backend that refers to the google cloud storage backend.
The service account I'm using in google has access to create,update,list,get files but NOT delete them.

This stops the rclone WebDAV server from "editing" a file - since rclone appears to want to delete the file during the WebDAV PUT operation. (Note that adding a file works fine, but replacing an existing one does not)

What is your rclone version (output from `rclone version`)

rclone v1.56.0

os/version: debian 10.10 (64 bit)
os/kernel: 5.4.72-microsoft-standard-WSL2 (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.16.5
go/linking: static
go/tags: none

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Linux docker image on Windows 10 64 Bit.

Which cloud storage system are you using? (eg Google Drive)

google cloud storage

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

      - serve 
      - webdav 
      - --addr
      - :8080
      - --auth-proxy 
      - /usr/local/bin/node /root/auth-proxy.mjs
      - --log-level
      - DEBUG
      - --delete-excluded

The rclone config contents with secrets removed.

[gcs-admin]
type = google cloud storage
project_number = *****
service_account_file = ./gcs/sa-admin.json
object_acl = projectPrivate
bucket_acl = projectPrivate
bucket_policy_only = true
location = us-east4

[gcs-reader]
type = google cloud storage
project_number = *****
service_account_file = ./gcs/sa-reader.json
object_acl = projectPrivate
bucket_acl = projectPrivate
bucket_policy_only = true
location = us-east4

[gcs-writer]
type = google cloud storage
project_number = *****
service_account_file = ./gcs/sa-writer.json
object_acl = projectPrivate
bucket_acl = projectPrivate
bucket_policy_only = true
location = us-east4

[gcs-updater]
type = google cloud storage
project_number = *****
service_account_file = ./gcs/sa-updater.json
object_acl = projectPrivate
bucket_acl = projectPrivate
bucket_policy_only = true
location = us-east4

A log from the command with the `-vv` flag

webdav-proxy_1  | 2021/09/24 17:41:40 DEBUG : GCS bucket <bucket name> path <file path>: File to upload is small (5186 bytes), uploading instead of streaming
webdav-proxy_1  | 2021/09/24 17:41:40 ERROR : <filename>: Failed to copy: googleapi: Error 403: <saaccount>.iam.gserviceaccount.com does not have storage.objects.delete access to the Google Cloud Storage object., forbidden
webdav-proxy_1  | 2021/09/24 17:41:40 ERROR : <filename>: WriteFileHandle.New Rcat failed: googleapi: Error 403: <saaccount>.iam.gserviceaccount.com does not have storage.objects.delete access to the Google Cloud Storage object., forbidden
webdav-proxy_1  | 2021/09/24 17:41:40 INFO  : /<filename>: PUT from 172.23.0.1:47830

asdffdsa · September 24, 2021, 9:02pm

hello and welcome to the forum,

i did a quick test trying to use your use-case, i could not reproduce the issue.
i used a aws s3 clone called wasabi and the bucket does not have s3:DeleteObject.

i was able to re-edit an existing file.

hard to know what is going on without a full debug log...
as a test, re-start the rclone serve webdav, re-edit that same text file, kill the webdav server.
that should create a small debug log, post that entire log.

another option, which will produce a larger debug log is to use
--dump=headers --retries=1 --low-level-retries=1 --log-level=DEBUG --log-file=rclone.log

ncw · September 26, 2021, 12:38pm

What webdav client are you using?

From the error messages it looks like rclone is trying to delete the file first but it would be good to check the HTTP status, so can you run with -vv --dump headers then we can see exactly what commands rclone is sending to GCS and whether it is sending a Delete, or whether GCS is returning the Delete from the upload request.

From checking the code I think we'll find rclone isn't sending a delete...

You could also try rclone copy to overwrite an existing file and see if you get the same error. If you do then that would be easier to debug.

awoodobvio · September 27, 2021, 5:50pm

I don't think this is an Rclone specific issue after looking into it more, but rather a GCS one.

Here is a relevant issue from stack overflow:
Why does gsutil cp require storage.objects.delete on versioned bucket? - Stack Overflow

Thanks for the additional command line (--dump headers). That let me see what was actually happening.
The logs below (with redacted information) from rclone copy clearly show it is executing the POST command to GCS to start the upload and that causes google to send back the 403 error code.

Therefore, it is a counter-intuitive thing that overwriting a file requires "delete" permission in GCS and there is no way to build a service account that can "update" but not "delete" objects in GCS.

Thanks for the assist to get the to the bottom of the issue!

2021/09/27 17:40:09 DEBUG : rclone: Version "v1.56.0" starting with parameters ["rclone" "-vv" "--dump" "headers" "copy" "hi.txt" "gcs-updater:/<bucket>/<directory>/hi.txt"]
2021/09/27 17:40:09 DEBUG : Creating backend with remote "hi.txt"
2021/09/27 17:40:09 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2021/09/27 17:40:09 DEBUG : fs cache: adding new entry for parent of "hi.txt", "/root"
2021/09/27 17:40:09 DEBUG : Creating backend with remote "gcs-updater:/<bucket>/<directory>/hi.txt"
2021/09/27 17:40:09 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : HTTP REQUEST (req 0xc00064dd00)
2021/09/27 17:40:09 DEBUG : POST /token HTTP/1.1
Host: oauth2.googleapis.com
User-Agent: rclone/v1.56.0
Content-Length: 795
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip

2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 DEBUG : HTTP RESPONSE (req 0xc00064dd00)
2021/09/27 17:40:09 DEBUG : HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Mon, 27 Sep 2021 17:40:06 GMT
Server: scaffolding on HTTPServer2
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : HTTP REQUEST (req 0xc000708300)
2021/09/27 17:40:09 DEBUG : GET /storage/v1/b/<bucket>/o/<directory>%2Fhi.txt?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.56.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.16.6 gdcl/20210406
Accept-Encoding: gzip

2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 DEBUG : HTTP RESPONSE (req 0xc000708300)
2021/09/27 17:40:09 DEBUG : HTTP/2.0 404 Not Found
Content-Length: 279
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Mon, 27 Sep 2021 17:40:06 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: <redacted>

2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 DEBUG : fs cache: renaming cache item "gcs-updater:/<bucket>/<directory>/hi.txt" to be canonical "gcs-updater:<bucket>/<directory>/hi.txt"
2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : HTTP REQUEST (req 0xc000199500)
2021/09/27 17:40:09 DEBUG : GET /storage/v1/b/<bucket>/o/<directory>%2Fhi.txt%2Fhi.txt?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.56.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.16.6 gdcl/20210406
Accept-Encoding: gzip

2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 DEBUG : HTTP RESPONSE (req 0xc000199500)
2021/09/27 17:40:09 DEBUG : HTTP/2.0 200 OK
Content-Length: 1000
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Mon, 27 Sep 2021 17:40:06 GMT
Etag: COOVqYXYn/MCEAE=
Expires: Mon, 27 Sep 2021 17:40:06 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: <redacted>

2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 DEBUG : hi.txt: Sizes differ (src 5 vs dst 3)
2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : HTTP REQUEST (req 0xc000199d00)
2021/09/27 17:40:09 DEBUG : GET /storage/v1/b/<bucket>/o?alt=json&maxResults=1&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.56.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.16.6 gdcl/20210406
Accept-Encoding: gzip

2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 DEBUG : HTTP RESPONSE (req 0xc000199d00)
2021/09/27 17:40:09 DEBUG : HTTP/2.0 200 OK
Content-Length: 1037
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Mon, 27 Sep 2021 17:40:07 GMT
Expires: Mon, 27 Sep 2021 17:40:07 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: <redacted>

2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : HTTP REQUEST (req 0xc0002b7800)
2021/09/27 17:40:09 DEBUG : POST /upload/storage/v1/b/<bucket>/o?alt=json&name=<directory>%2Fhi.txt%2Fhi.txt&prettyPrint=false&uploadType=multipart HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.56.0
Transfer-Encoding: chunked
Authorization: XXXX
Content-Type: multipart/related; boundary=80d31d8620982f8345626a0ef4e1e41f98ff371706a6928cc23c04fa1318
X-Goog-Api-Client: gl-go/1.16.6 gdcl/20210406
Accept-Encoding: gzip

2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 DEBUG : HTTP RESPONSE (req 0xc0002b7800)
2021/09/27 17:40:09 DEBUG : HTTP/2.0 403 Forbidden
Content-Length: 366
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Mon, 27 Sep 2021 17:40:07 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: <redacted>

2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 ERROR : hi.txt: Failed to copy: googleapi: Error 403: rclone-updater@<projectid>.iam.gserviceaccount.com does not have storage.objects.delete access to the Google Cloud Storage object., forbidden
2021/09/27 17:40:09 ERROR : Attempt 1/3 failed with 1 errors and: googleapi: Error 403: rclone-updater@<projectid>.iam.gserviceaccount.com does not have storage.objects.delete access to the Google Cloud Storage object., forbidden
2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : HTTP REQUEST (req 0xc000708700)
2021/09/27 17:40:09 DEBUG : GET /storage/v1/b/<bucket>/o/<directory>%2Fhi.txt%2Fhi.txt?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.56.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.16.6 gdcl/20210406
Accept-Encoding: gzip

2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 DEBUG : HTTP RESPONSE (req 0xc000708700)
2021/09/27 17:40:09 DEBUG : HTTP/2.0 200 OK
Content-Length: 1000
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Mon, 27 Sep 2021 17:40:07 GMT
Etag: COOVqYXYn/MCEAE=
Expires: Mon, 27 Sep 2021 17:40:07 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: <redacted>

2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 DEBUG : hi.txt: Sizes differ (src 5 vs dst 3)
2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : HTTP REQUEST (req 0xc000708c00)
2021/09/27 17:40:09 DEBUG : POST /upload/storage/v1/b/<bucket>/o?alt=json&name=<directory>%2Fhi.txt%2Fhi.txt&prettyPrint=false&uploadType=multipart HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.56.0
Transfer-Encoding: chunked
Authorization: XXXX
Content-Type: multipart/related; boundary=59796b6ca9b43b1ed26b6360862b9a35d65dd17b9a5c7c1ccb9a1afe0bf5
X-Goog-Api-Client: gl-go/1.16.6 gdcl/20210406
Accept-Encoding: gzip

2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 DEBUG : HTTP RESPONSE (req 0xc000708c00)
2021/09/27 17:40:09 DEBUG : HTTP/2.0 403 Forbidden
Content-Length: 366
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Mon, 27 Sep 2021 17:40:07 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: <redacted>

2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 ERROR : hi.txt: Failed to copy: googleapi: Error 403: rclone-updater@<projectid>.iam.gserviceaccount.com does not have storage.objects.delete access to the Google Cloud Storage object., forbidden
2021/09/27 17:40:09 ERROR : Attempt 2/3 failed with 1 errors and: googleapi: Error 403: rclone-updater@<projectid>.iam.gserviceaccount.com does not have storage.objects.delete access to the Google Cloud Storage object., forbidden
2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : HTTP REQUEST (req 0xc000709100)
2021/09/27 17:40:09 DEBUG : GET /storage/v1/b/<bucket>/o/<directory>%2Fhi.txt%2Fhi.txt?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.56.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.16.6 gdcl/20210406
Accept-Encoding: gzip

2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 DEBUG : HTTP RESPONSE (req 0xc000709100)
2021/09/27 17:40:09 DEBUG : HTTP/2.0 200 OK
Content-Length: 1000
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Mon, 27 Sep 2021 17:40:07 GMT
Etag: COOVqYXYn/MCEAE=
Expires: Mon, 27 Sep 2021 17:40:07 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: <redacted>

2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 DEBUG : hi.txt: Sizes differ (src 5 vs dst 3)
2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : HTTP REQUEST (req 0xc00064dc00)
2021/09/27 17:40:09 DEBUG : POST /upload/storage/v1/b/<bucket>/o?alt=json&name=<directory>%2Fhi.txt%2Fhi.txt&prettyPrint=false&uploadType=multipart HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.56.0
Transfer-Encoding: chunked
Authorization: XXXX
Content-Type: multipart/related; boundary=6c7356bc76ff8b64c5d9799088deace9335d41106806b8ad0d7bea9ffc54
X-Goog-Api-Client: gl-go/1.16.6 gdcl/20210406
Accept-Encoding: gzip

2021/09/27 17:40:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 DEBUG : HTTP RESPONSE (req 0xc00064dc00)
2021/09/27 17:40:09 DEBUG : HTTP/2.0 403 Forbidden
Content-Length: 366
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Mon, 27 Sep 2021 17:40:07 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: <redacted>

2021/09/27 17:40:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/09/27 17:40:09 ERROR : hi.txt: Failed to copy: googleapi: Error 403: rclone-updater@<projectid>.iam.gserviceaccount.com does not have storage.objects.delete access to the Google Cloud Storage object., forbidden
2021/09/27 17:40:09 ERROR : Attempt 3/3 failed with 1 errors and: googleapi: Error 403: rclone-updater@<projectid>.iam.gserviceaccount.com does not have storage.objects.delete access to the Google Cloud Storage object., forbidden
2021/09/27 17:40:09 INFO  : 
Transferred:   	         15 / 15 Byte, 100%, 0 Byte/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.5s

2021/09/27 17:40:09 DEBUG : 5 go routines active
2021/09/27 17:40:09 Failed to copy: googleapi: Error 403: rclone-updater@<projectid>.iam.gserviceaccount.com does not have storage.objects.delete access to the Google Cloud Storage object., forbidden

ncw · September 28, 2021, 12:00pm

Well done for finding the answer.

That SO answer seems quite clear - you are going to need the delete permission too for versioned buckets.

system · October 28, 2021, 12:00pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.