I was Unable configure AWS s3 with Rclone

Help and Support
1

What is the problem you are having with rclone?

I have configured my AWS s3 in rclone config section and trying to create directory using

rclone lsd AWSMoodagent:

error: The specified bucket does not have a website configuration

rclone mkdir AWSMoodagent:musicstorys3

And I am getting the bellow error:

caused by: expected element type but have
2020/07/07 12:11:26 ERROR : Attempt 3/3 failed with 1 errors and: SerializationError: failed to unmarshal error message
status code: 404, request id: 1781B9741CCF141F, host id: lYbxKzf8yCXGIuy/uonghZHMWWCC7h4amZ0/e5Uc/kYvfAHBE0UIPaointzzURU1x/DZ9LuDgzM=
caused by: UnmarshalError: failed to unmarshal error message
00000000 3c 68 74 6d 6c 3e 0a 3c 68 65 61 64 3e 3c 74 69 |.<ti|
00000010 74 6c 65 3e 34 30 34 20 4e 6f 74 20 46 6f 75 6e |tle>404 Not Foun|
00000020 64 3c 2f 74 69 74 6c 65 3e 3c 2f 68 65 61 64 3e |d|
00000030 0a 3c 62 6f 64 79 3e 0a 3c 68 31 3e 34 30 34 20 |..

404 |
00000040 4e 6f 74 20 46 6f 75 6e 64 3c 2f 68 31 3e 0a 3c |Not Found

.<|
00000050 75 6c 3e 0a 3c 6c 69 3e 43 6f 64 65 3a 20 4e 6f |ul>.
  • Code: No|
    00000060 53 75 63 68 42 75 63 6b 65 74 3c 2f 6c 69 3e 0a |SuchBucket
    • .|
    00000070 3c 6c 69 3e 4d 65 73 73 61 67 65 3a 20 54 68 65 |
  • Message: The|
    00000080 20 73 70 65 63 69 66 69 65 64 20 62 75 63 6b 65 | specified bucke|
    00000090 74 20 64 6f 65 73 20 6e 6f 74 20 65 78 69 73 74 |t does not exist|
    000000a0 3c 2f 6c 69 3e 0a 3c 6c 69 3e 42 75 63 6b 65 74 |
    • .
  • Bucket|
    000000b0 4e 61 6d 65 3a 20 6d 6f 6f 64 61 67 65 6e 74 2d |Name: moodagent-|
    000000c0 6d 75 73 69 63 73 74 6f 72 79 2d 74 65 73 74 2e |musicstory-test.|
    000000d0 6d 6f 6f 64 61 67 65 6e 74 2d 6d 75 73 69 63 73 |moodagent-musics|
    000000e0 74 6f 72 79 2d 74 65 73 74 3c 2f 6c 69 3e 0a 3c |tory-test
    • .<|
    000000f0 6c 69 3e 52 65 71 75 65 73 74 49 64 3a 20 31 37 |li>RequestId: 17|
    00000100 38 31 42 39 37 34 31 43 43 46 31 34 31 46 3c 2f |81B9741CCF141F</|
    00000110 6c 69 3e 0a 3c 6c 69 3e 48 6f 73 74 49 64 3a 20 |li>.
  • HostId: |
    00000120 6c 59 62 78 4b 7a 66 38 79 43 58 47 49 75 79 2f |lYbxKzf8yCXGIuy/|
    00000130 75 6f 6e 67 68 5a 48 4d 57 57 43 43 37 68 34 61 |uonghZHMWWCC7h4a|
    00000140 6d 5a 30 2f 65 35 55 63 2f 6b 59 76 66 41 48 42 |mZ0/e5Uc/kYvfAHB|
    00000150 45 30 55 49 50 61 6f 69 6e 74 7a 7a 55 52 55 31 |E0UIPaointzzURU1|
    00000160 78 2f 44 5a 39 4c 75 44 67 7a 4d 3d 3c 2f 6c 69 |x/DZ9LuDgzM=</li|
    00000170 3e 0a 3c 2f 75 6c 3e 0a 3c 68 72 2f 3e 0a 3c 2f |>..
    .</|
    00000180 62 6f 64 79 3e 0a 3c 2f 68 74 6d 6c 3e 0a |body>..|

    What is your rclone version (output from rclone version)

    rclone v1.52.2

    • os/arch: darwin/amd64
    • go version: go1.14.4

    Which cloud storage system are you using? (eg Google Drive)

    AWS S3

    The command you were trying to run (eg rclone copy /tmp remote:tmp)

    rclone lsd AWSMoodagent:
    rclone mkdir AWSMoodagent:musicstorys3

    The rclone config contents with secrets removed.

    "provider" = "AWS"
    "env_auth" = "false"
    "access_key_id" = "my access key"
    "secret_access_key" = " mysecret key "
    "region" = "us-east-1"
    "endpoint" = "my aws end point url"
    "location_constraint" = ""
    "acl" = "private"
    "server_side_encryption" = ""
    "sse_kms_key_id" = ""
    "storage_class" = "STANDARD"

    Any help on this would be really helpful.

    • 2

    You shouldn't need to provide this - try removing it.

    You error looks like HTML returned when rclone is expecting XML so the endpoint being wrong would make sense.

    3

    Thanks Nick :slight_smile:
    I figured out the issue and able to connect to AWS S3.

    Could you please help me with the way to avoid statically passing the AWS credentials while configuring the rclone AWS connection.

    And could you please also help me with one suitable docker image for working with rclone commands.

    Any help on this would be much appreciated :)'

    4

    Rclone can read AWS credentials from the environment: https://rclone.org/s3/#authentication

    So if you are running on ec2 then you can have the credentials appear there automatically.

    Otherwise I'm not sure what you mean?

    Here is the docs on installing and running with docker: https://rclone.org/install/#install-with-docker

    5

    This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.