Hi @asdffdsa and thank you for welcoming me!
- version:
rclone v1.54.0 - it will fail exactly for the reason described in this ticket - apparently RCLONE tries to create the bucket without listing first and RMT2 doesn't have bucket create permissions (which aren't really necessary).
- As I said, this is exactly as described by OP. The used account does not have CreateBucket permissions and it fails when attempting to copy a single file.
Working Command DEBUG:
╰─$ rclone copy -vv bla/ s3-placer-guy-test:guyarad-test
2021/04/23 09:27:49 DEBUG : rclone: Version "v1.54.0" starting with parameters ["rclone" "copy" "-vv" "bla/" "s3-placer-guy-test:guyarad-test"]
2021/04/23 09:27:49 DEBUG : Creating backend with remote "bla/"
2021/04/23 09:27:49 DEBUG : Using config file from "/Users/guyarad/.config/rclone/rclone.conf"
2021/04/23 09:27:49 DEBUG : fs cache: renaming cache item "bla/" to be canonical "/Users/guyarad/src/placer-django-server/bla"
2021/04/23 09:27:49 DEBUG : Creating backend with remote "s3-placer-guy-test:guyarad-test"
2021/04/23 09:27:50 DEBUG : S3 bucket guyarad-test: Waiting for checks to finish
2021/04/23 09:27:50 DEBUG : S3 bucket guyarad-test: Waiting for transfers to finish
2021/04/23 09:27:51 DEBUG : bla.txt: MD5 = 3cd7a0db76ff9dca48979e24c39b408c OK
2021/04/23 09:27:51 INFO : bla.txt: Copied (new)
2021/04/23 09:27:51 INFO :
Failed Command DEBUG:
╰─$ rclone copy -vvv bla/bla.txt s3-placer-guy-test:guyarad-test
2021/04/23 09:23:27 DEBUG : rclone: Version "v1.54.0" starting with parameters ["rclone" "copy" "-vvv" "bla/bla.txt" "s3-placer-guy-test:guyarad-test"]
2021/04/23 09:23:27 DEBUG : Creating backend with remote "bla/bla.txt"
2021/04/23 09:23:27 DEBUG : Using config file from "/Users/guyarad/.config/rclone/rclone.conf"
2021/04/23 09:23:27 DEBUG : fs cache: adding new entry for parent of "bla/bla.txt", "/Users/guyarad/src/placer-django-server/bla"
2021/04/23 09:23:27 DEBUG : Creating backend with remote "s3-placer-guy-test:guyarad-test"
2021/04/23 09:23:27 DEBUG : bla.txt: Need to transfer - File not found at Destination
2021/04/23 09:23:28 ERROR : bla.txt: Failed to copy: AccessDenied: Access Denied
status code: 403, request id: K64RGDM8SXYWN039, host id: iLEI5TtvmJNOOmTMjhr17Zv7W6H7kc5Oxzbq3V9/072+dOC6XPhzdJqjqM0hQ+sfKr3YS0HDmLk=
2021/04/23 09:23:28 ERROR : Attempt 1/3 failed with 1 errors and: AccessDenied: Access Denied
status code: 403, request id: K64RGDM8SXYWN039, host id: iLEI5TtvmJNOOmTMjhr17Zv7W6H7kc5Oxzbq3V9/072+dOC6XPhzdJqjqM0hQ+sfKr3YS0HDmLk=
2021/04/23 09:23:28 DEBUG : bla.txt: Need to transfer - File not found at Destination
2021/04/23 09:23:28 ERROR : bla.txt: Failed to copy: AccessDenied: Access Denied
After adding CreateBucket policy for the user, this is output (notice the "bucket created"):
2021/04/23 09:41:00 DEBUG : Creating backend with remote "bla/bla.txt"
2021/04/23 09:41:00 DEBUG : Using config file from "/Users/guyarad/.config/rclone/rclone.conf"
2021/04/23 09:41:00 DEBUG : fs cache: adding new entry for parent of "bla/bla.txt", "/Users/guyarad/src/placer-django-server/bla"
2021/04/23 09:41:00 DEBUG : Creating backend with remote "s3-placer-guy-test:guyarad-test"
2021/04/23 09:41:00 DEBUG : bla.txt: Need to transfer - File not found at Destination
2021/04/23 09:41:01 INFO : S3 bucket guyarad-test: Bucket "guyarad-test" created with ACL "private"
2021/04/23 09:41:01 DEBUG : bla.txt: MD5 = 3cd7a0db76ff9dca48979e24c39b408c OK
2021/04/23 09:41:01 INFO : bla.txt: Copied (new)