# Rclone attempts to create an S3 bucket even though the bucket and path already exist

**URL:** <https://forum.rclone.org/t/rclone-attempts-to-create-an-s3-bucket-even-though-the-bucket-and-path-already-exist/49370>\
**Category:** Help and Support\
**Created:** [January 4, 2025, 6:11pm UTC](https://forum.rclone.org/t/rclone-attempts-to-create-an-s3-bucket-even-though-the-bucket-and-path-already-exist/49370 "2025-01-04T18:11:00Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Voyager](https://forum.rclone.org/letter_avatar_proxy/v4/letter/v/46a35a/32.png) [@Voyager](https://forum.rclone.org/u/Voyager)\
**Post date:** [January 4, 2025, 6:11pm UTC](https://forum.rclone.org/t/rclone-attempts-to-create-an-s3-bucket-even-though-the-bucket-and-path-already-exist/49370/1 "2025-01-04T18:11:00Z")

</div>

#### What is the problem you are having with rclone?

I'm trying to copy files to an existing S3 bucket but I get an error. The bucket already exists and other copy jobs are working as expected. In the config below, `cryptbucket` works as expected.

IAM policy for the credentials that rclone are using have `PutObject` rights to the path in question and does not have permissions to `CreateBucket` intentionally and would like to keep it that way.

I've gotten this error a number of times and often it's not immediately clear as to why I get it. I'm assuming rclone is trying to perform some sort of check and when it fails, it assumes the bucket doesn't exist and attempts to create it.

(aside: I've gotten this error when performing a copy using the cryptbucket remote with `--no-traverse` option. Removing the option resolved the issue.)

#### Run the command 'rclone version' and share the full output of the command.

```auto
$ rclone version
rclone v1.68.2
- os/version: linuxmint 21.3 (64 bit)
- os/kernel: 6.8.0-50-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.23.3
- go/linking: static
- go/tags: none

```

#### Which cloud storage system are you using? (eg Google Drive)

AWS S3

#### The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

```auto
rclone copyto -vv --header-upload "x-amz-tagging: backup-ring=0&backup-home-<<USERNAME>>=2025-01" --interactive /path/to/home-<<USERNAME>>-level0.tar.gz home-<<USERNAME>>:/<<HOSTNAME>>

```

#### Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

```auto
[aws-s3]
type = s3
provider = AWS
env_auth = true
access_key_id = XXX
secret_access_key = XXX
region = us-east-1
server_side_encryption = AES256

[cryptbucket]
type = crypt
remote = aws-s3:<<REDACTED>>/archives/<<HOSTNAME>>/<<PERSONAL_DIRECTORY>>
password = XXX

[home-<<USERNAME>>]
type = alias
remote = aws-s3:<<REDACTED>>/archives/<<USERNAME>>
description = AWS S3 path for <<USERNAME>>

```

Please note that `cryptbucket` works fine. `home-<<USERNAME>>` is NOT a crypt remote and uses an alias to make typing into the console a bit easier. I have tried running the the copy command against the `aws-s3:<<BUCKET>>/archives/<<USERNAME>>` and got the same error.

#### A log from the command that you were trying to run with the `-vv` flag

```auto
$ rclone copyto -vv --header-upload "x-amz-tagging: backup-ring=0&backup-home-<<USERNAME>>=2025-01" --interactive /path/to/home-<<USERNAME>>-level0.tar.gz home-<<USERNAME>>:/<<HOSTNAME>>
2025/01/04 12:40:03 DEBUG : rclone: Version "v1.68.2" starting with parameters ["rclone" "copyto" "-vv" "--header-upload" "x-amz-tagging: backup-ring=0&backup-home-<<USERNAME>>=2025-01" "--interactive" "/path/to/home-<<USERNAME>>-level0.tar.gz" "home-<<USERNAME>>:/<<HOSTNAME>>"]
2025/01/04 12:40:03 DEBUG : Creating backend with remote "/path/to/home-<<USERNAME>>-level0.tar.gz"
2025/01/04 12:40:03 DEBUG : Using config file from "/home/<<USERNAME>>/.config/rclone/rclone.conf"
2025/01/04 12:40:03 DEBUG : fs cache: adding new entry for parent of "/path/to/home-<<USERNAME>>-level0.tar.gz", "/path/to"
2025/01/04 12:40:03 DEBUG : Creating backend with remote "home-<<USERNAME>>:/"
2025/01/04 12:40:03 DEBUG : Creating backend with remote "aws-s3:<<S3-BUCKET>>/archives/michael.soh"
2025/01/04 12:40:03 DEBUG : fs cache: renaming cache item "home-<<USERNAME>>:/" to be canonical "aws-s3:<<S3-BUCKET>>/archives/michael.soh"
2025/01/04 12:40:03 DEBUG : home-<<USERNAME>>-level0.tar.gz: Need to transfer - File not found at Destination

rclone: copy "home-<<USERNAME>>-level0.tar.gz"?
y) Yes, this is OK (default)
n) No, skip this
s) Skip all copy operations with no more questions
!) Do all copy operations with no more questions
q) Exit rclone now.
y/n/s/!/q> y
2025/01/04 12:40:06 DEBUG : home-<<USERNAME>>-level0.tar.gz: multi-thread copy: disabling buffering because source is local disk
2025/01/04 12:40:06 ERROR : home-<<USERNAME>>-level0.tar.gz: Failed to copy: multi-thread copy: failed to open chunk writer: failed to prepare upload: operation error S3: CreateBucket, https response error StatusCode: 403, RequestID: YMTJ56YZVGRDKR97, HostID: U0khycZoH/gRB7NadLa3FZzbcnrvsHQTBKCR7b0I+8pvcX02Mc5qvO/i0HitkmOSLr7Y9xNz3ak=, api error AccessDenied: User: arn:aws:iam::<<AWS-ACCOUNT-ID>>:user/rclone-user is not authorized to perform: s3:CreateBucket on resource: "arn:aws:s3:::<<S3-BUCKET>>" because no identity-based policy allows the s3:CreateBucket action
2025/01/04 12:40:06 ERROR : Attempt 1/3 failed with 1 errors and: multi-thread copy: failed to open chunk writer: failed to prepare upload: operation error S3: CreateBucket, https response error StatusCode: 403, RequestID: YMTJ56YZVGRDKR97, HostID: U0khycZoH/gRB7NadLa3FZzbcnrvsHQTBKCR7b0I+8pvcX02Mc5qvO/i0HitkmOSLr7Y9xNz3ak=, api error AccessDenied: User: arn:aws:iam::<<AWS-ACCOUNT-ID>>:user/rclone-user is not authorized to perform: s3:CreateBucket on resource: "arn:aws:s3:::<<S3-BUCKET>>" because no identity-based policy allows the s3:CreateBucket action
2025/01/04 12:40:06 DEBUG : home-<<USERNAME>>-level0.tar.gz: Need to transfer - File not found at Destination

```

---

<div class="post-metadata">

**Author:** ![asdffdsa](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/848f3c/32.png) [@asdffdsa](https://forum.rclone.org/u/asdffdsa)\
**Post date:** [January 4, 2025, 6:38pm UTC](https://forum.rclone.org/t/rclone-attempts-to-create-an-s3-bucket-even-though-the-bucket-and-path-already-exist/49370/2 "2025-01-04T18:38:21Z")

</div>

> [@Voyager](#):
>
> does not have permissions to CreateBucket intentionally and would like to keep it that way.

try `--s3-no-check-bucket`

---

<div class="post-metadata">

**Author:** ![Voyager](https://forum.rclone.org/letter_avatar_proxy/v4/letter/v/46a35a/32.png) [@Voyager](https://forum.rclone.org/u/Voyager)\
**Post date:** [January 4, 2025, 7:39pm UTC](https://forum.rclone.org/t/rclone-attempts-to-create-an-s3-bucket-even-though-the-bucket-and-path-already-exist/49370/3 "2025-01-04T19:39:15Z")

</div>

Thanks! I remembered vaugely that there was an option to prevent this check but couldn't remember it.

That said, any reason why the rclone attempts to create the bucket? The docs says:

> If set, don't attempt to check the bucket exists or create it.

If rclone checks the bucket exists, it should pass and thus `CreateBucket` wouldn't need to be called.

I'm still pretty new to rclone so I'm not 100% sure how the workflow works.

---

<div class="post-metadata">

**Author:** ![asdffdsa](https://forum.rclone.org/letter_avatar_proxy/v4/letter/a/848f3c/32.png) [@asdffdsa](https://forum.rclone.org/u/asdffdsa)\
**Post date:** [January 6, 2025, 8:26pm UTC](https://forum.rclone.org/t/rclone-attempts-to-create-an-s3-bucket-even-though-the-bucket-and-path-already-exist/49370/4 "2025-01-06T20:26:20Z")

</div>

> [@Voyager](#):
>
> That said, any reason why the rclone attempts to create the bucket?

before rclone copies a file, it wants to confirm that the bucket exists.

> [@Voyager](#):
>
> If rclone checks the bucket exists, it should pass and thus `CreateBucket` wouldn't need to be called.

there are subtle differences between `rclone copy` and `rclone copyto`  
fwiw, always use `rclone copy` unless there is a very specific reason to use `rclone copyto` ?

---

<div class="post-metadata">

**Author:** ![system](https://forum.rclone.org/uploads/default/original/2X/3/39ad3235f6ffec76ccd0c81616bd55697b4c300d.png) [@system](https://forum.rclone.org/u/system)\
**Post date:** [January 9, 2025, 8:27pm UTC](https://forum.rclone.org/t/rclone-attempts-to-create-an-s3-bucket-even-though-the-bucket-and-path-already-exist/49370/5 "2025-01-09T20:27:11Z")

</div>

Closed on January 9, 2025, 8:27pm UTC

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.
