What is the problem you are having with rclone?
Identical to the problem reported here except the proposed solution does not work for me.
Briefly:
We have permissions set up so that a specific user account needs to use the "bucket-owner-full-control" ACL in order to be able to write to a specific S3 bucket.
If I do that in the AWS CLI, it works:
$ aws s3 cp hi s3://fh-pi-holland-e-eco/test/test.txt --acl bucket-owner-full-control
upload: ./hi to s3://fh-pi-holland-e-eco/test/test.txt
$
I tried the solution proposed in the above-linked GitHub issue, but it did not work. See below.
Run the command 'rclone version' and share the full output of the command.
$ rclone version
rclone v1.64.0
- os/version: ubuntu 18.04 (64 bit)
- os/kernel: 4.15.0-192-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.1
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
AWS S3
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone -vv copyto ./hi s3:fh-pi-holland-e-eco/test/test.txt
The rclone config contents with secrets removed.
[s3]
type = s3
env_auth = true
region = us-west-2
provider = AWS
acl = bucket-owner-full-control
location_constraint = us-west-2
server_side_encryption = aws:kms
A log from the command with the -vv flag
2023/09/15 10:29:18 DEBUG : rclone: Version "v1.64.0" starting with parameters ["rclone" "-vv" "copyto" "./hi" "s3:fh-pi-holland-e-eco/test/test.txt"]
2023/09/15 10:29:18 DEBUG : Creating backend with remote "./hi"
2023/09/15 10:29:18 DEBUG : Using config file from "/home/dtenenba/.rclone.conf"
2023/09/15 10:29:18 DEBUG : fs cache: adding new entry for parent of "./hi", "/home/dtenenba"
2023/09/15 10:29:18 DEBUG : Creating backend with remote "s3:fh-pi-holland-e-eco/test/"
2023/09/15 10:29:18 DEBUG : fs cache: renaming cache item "s3:fh-pi-holland-e-eco/test/" to be canonical "s3:fh-pi-holland-e-eco/test"
2023/09/15 10:29:19 DEBUG : hi: Need to transfer - File not found at Destination
2023/09/15 10:29:19 ERROR : hi: Failed to copy: failed to prepare upload: AccessDenied: Access Denied
status code: 403, request id: 0V2P95A15S45P2KG, host id: LN8sj7+SCn8TMXr+QE/bb9l/btzcHIEYNrSFLf4XlAPY9V9Z/WslRQ3fTFRxx7PlYz+jr8O0Of5Bze61Jlkffw==
2023/09/15 10:29:19 ERROR : Attempt 1/3 failed with 1 errors and: failed to prepare upload: AccessDenied: Access Denied
status code: 403, request id: 0V2P95A15S45P2KG, host id: LN8sj7+SCn8TMXr+QE/bb9l/btzcHIEYNrSFLf4XlAPY9V9Z/WslRQ3fTFRxx7PlYz+jr8O0Of5Bze61Jlkffw==
2023/09/15 10:29:19 DEBUG : hi: Need to transfer - File not found at Destination
2023/09/15 10:29:19 ERROR : hi: Failed to copy: failed to prepare upload: AccessDenied: Access Denied
status code: 403, request id: 0V2VA11RHQ2VK59M, host id: asQOEBm3V09G1heItuteIS6/Di2VvNJIVZ7I9yp+ehcQ7AxZb/Pe8YXapBc6qsGLtRJCQKwsg9o=
2023/09/15 10:29:19 ERROR : Attempt 2/3 failed with 1 errors and: failed to prepare upload: AccessDenied: Access Denied
status code: 403, request id: 0V2VA11RHQ2VK59M, host id: asQOEBm3V09G1heItuteIS6/Di2VvNJIVZ7I9yp+ehcQ7AxZb/Pe8YXapBc6qsGLtRJCQKwsg9o=
2023/09/15 10:29:19 DEBUG : hi: Need to transfer - File not found at Destination
2023/09/15 10:29:19 ERROR : hi: Failed to copy: failed to prepare upload: AccessDenied: Access Denied
status code: 403, request id: 0V2P1MW8JGMBVYMD, host id: 46hIBBarcyeYoAxD45zRRohKZuwl8ubjnk3Fkj9c+KpqKPMzsto/ORjaEKHcyaC/oLFniSAekHw=
2023/09/15 10:29:19 ERROR : Attempt 3/3 failed with 1 errors and: failed to prepare upload: AccessDenied: Access Denied
status code: 403, request id: 0V2P1MW8JGMBVYMD, host id: 46hIBBarcyeYoAxD45zRRohKZuwl8ubjnk3Fkj9c+KpqKPMzsto/ORjaEKHcyaC/oLFniSAekHw=
2023/09/15 10:29:19 INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.3s
2023/09/15 10:29:19 DEBUG : 5 go routines active
2023/09/15 10:29:19 Failed to copyto: failed to prepare upload: AccessDenied: Access Denied
status code: 403, request id: 0V2P1MW8JGMBVYMD, host id: 46hIBBarcyeYoAxD45zRRohKZuwl8ubjnk3Fkj9c+KpqKPMzsto/ORjaEKHcyaC/oLFniSAekHw=
Also, although I understand that modifying the config file is the "correct" way to solve this issue, I would really prefer a solution that I can use on the command line (as part of the rclone command), because we have a GUI webapp built on top of rclone and it is set up to run without a config file, passing everything we need either as an environment variable or a command-line flag. Users do not necessarily have ~/.rclone.conf files or if they do, we can't count on them having the correct contents. So I really hope there is a way to make this work that does not depend on the config file.
FWIW, I get the same results when using --s3-acl bucket-owner-full-control.
Thank you!