Copy returns a error 400

Macario · October 15, 2019, 3:52am

What is the problem you are having with rclone?

When trying to copy files to my digitalocean space i get a Failed to copy: s3 upload: 400 Bad Request, but i can copy files from the space just fine.

What is your rclone version (output from `rclone version`)

I'm using rclone v1.49.5.

os/arch: linux/amd64
go version: go1.13.1

Which OS you are using and how many bits (eg Windows 7, 64 bit)

My os is Antergos Linux.

Which cloud storage system are you using? (eg Google Drive)

I am using Digital Ocean spaces.

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone copy mozilla.pdf digitalocean:sc-images/test_bucket

A log from the command with the `-vv` flag (eg output from `rclone -vv copy /tmp remote:tmp`)

2019/10/14 22:40:46 ERROR : mozilla.pdf: Failed to copy: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>sc-images</BucketName><RequestId>tx0000000000001c4a3629f-005da53fbe-23fcca-sfo2a</RequestId><HostId>23fcca-sfo2a-sfo</HostId></Error>
2019/10/14 22:40:46 ERROR : Attempt 1/3 failed with 3 errors and: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>sc-images</BucketName><RequestId>tx0000000000001c4a3629f-005da53fbe-23fcca-sfo2a</RequestId><HostId>23fcca-sfo2a-sfo</HostId></Error>
2019/10/14 22:40:47 ERROR : mozilla.pdf: Failed to copy: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>sc-images</BucketName><RequestId>tx0000000000001c4a363ed-005da53fbf-23fcca-sfo2a</RequestId><HostId>23fcca-sfo2a-sfo</HostId></Error>
2019/10/14 22:40:47 ERROR : Attempt 2/3 failed with 3 errors and: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>sc-images</BucketName><RequestId>tx0000000000001c4a363ed-005da53fbf-23fcca-sfo2a</RequestId><HostId>23fcca-sfo2a-sfo</HostId></Error>
2019/10/14 22:40:47 ERROR : mozilla.pdf: Failed to copy: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>sc-images</BucketName><RequestId>tx0000000000001c4758715-005da53fbf-23e283-sfo2a</RequestId><HostId>23e283-sfo2a-sfo</HostId></Error>
2019/10/14 22:40:47 ERROR : Attempt 3/3 failed with 3 errors and: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>sc-images</BucketName><RequestId>tx0000000000001c4758715-005da53fbf-23e283-sfo2a</RequestId><HostId>23e283-sfo2a-sfo</HostId></Error>
2019/10/14 22:40:47 Failed to copy with 3 errors: last error was: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>sc-images</BucketName><RequestId>tx0000000000001c4758715-005da53fbf-23e283-sfo2a</RequestId><HostId>23e283-sfo2a-sfo</HostId></Error>

rclone copy mozilla.pdf digitalocean:sc-images/test_bucket --log-level DEBUG output

2019/10/14 23:23:40 DEBUG : rclone: Version "v1.49.5" starting with parameters ["rclone" "copy" "mozilla.pdf" "digitalocean:sc-images/test_bucket" "--log-level" "DEBUG"]
2019/10/14 23:23:40 DEBUG : Using config file from "/home/bruno/.config/rclone/rclone.conf"
2019/10/14 23:23:40 DEBUG : mozilla.pdf: Couldn't find file - need to transfer
2019/10/14 23:23:41 ERROR : mozilla.pdf: Failed to copy: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>sc-images</BucketName><RequestId>tx0000000000001c48a8a56-005da549cd-23e283-sfo2a</RequestId><HostId>23e283-sfo2a-sfo</HostId></Error>
2019/10/14 23:23:41 ERROR : Attempt 1/3 failed with 3 errors and: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>sc-images</BucketName><RequestId>tx0000000000001c48a8a56-005da549cd-23e283-sfo2a</RequestId><HostId>23e283-sfo2a-sfo</HostId></Error>
2019/10/14 23:23:41 DEBUG : mozilla.pdf: Couldn't find file - need to transfer
2019/10/14 23:23:41 DEBUG : pacer: low level retry 1/1 (error s3 upload: 503 Slow Down: <?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>SlowDown</Code>
<Message>Please reduce your request rate.</Message>
<RequestId></RequestId>
</Error>
)
2019/10/14 23:23:41 DEBUG : pacer: Rate limited, increasing sleep to 10ms
2019/10/14 23:23:41 DEBUG : mozilla.pdf: Received error: s3 upload: 503 Slow Down: <?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>SlowDown</Code>
<Message>Please reduce your request rate.</Message>
<RequestId></RequestId>
</Error>
 - low level retry 1/10
2019/10/14 23:23:42 DEBUG : pacer: low level retry 1/1 (error s3 upload: 503 Slow Down: <?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>SlowDown</Code>
<Message>Please reduce your request rate.</Message>
<RequestId></RequestId>
</Error>
)
2019/10/14 23:23:42 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2019/10/14 23:23:42 DEBUG : mozilla.pdf: Received error: s3 upload: 503 Slow Down: <?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>SlowDown</Code>
<Message>Please reduce your request rate.</Message>
<RequestId></RequestId>
</Error>
 - low level retry 2/10
2019/10/14 23:23:42 DEBUG : pacer: Reducing sleep to 15ms
2019/10/14 23:23:42 ERROR : mozilla.pdf: Failed to copy: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>sc-images</BucketName><RequestId>tx0000000000001c48a8c3d-005da549ce-23e283-sfo2a</RequestId><HostId>23e283-sfo2a-sfo</HostId></Error>
2019/10/14 23:23:42 ERROR : Attempt 2/3 failed with 3 errors and: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>sc-images</BucketName><RequestId>tx0000000000001c48a8c3d-005da549ce-23e283-sfo2a</RequestId><HostId>23e283-sfo2a-sfo</HostId></Error>
2019/10/14 23:23:42 DEBUG : pacer: Reducing sleep to 11.25ms
2019/10/14 23:23:42 DEBUG : mozilla.pdf: Couldn't find file - need to transfer
2019/10/14 23:23:43 DEBUG : pacer: Reducing sleep to 0s
2019/10/14 23:23:43 ERROR : mozilla.pdf: Failed to copy: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>sc-images</BucketName><RequestId>tx0000000000001c4b798e1-005da549cf-23fcca-sfo2a</RequestId><HostId>23fcca-sfo2a-sfo</HostId></Error>
2019/10/14 23:23:43 ERROR : Attempt 3/3 failed with 3 errors and: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>sc-images</BucketName><RequestId>tx0000000000001c4b798e1-005da549cf-23fcca-sfo2a</RequestId><HostId>23fcca-sfo2a-sfo</HostId></Error>
2019/10/14 23:23:43 Failed to copy with 3 errors: last error was: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>sc-images</BucketName><RequestId>tx0000000000001c4b798e1-005da549cf-23fcca-sfo2a</RequestId><HostId>23fcca-sfo2a-sfo</HostId></Error>

Here is my config file

[digitalocean]
type=s3
env_auth = false
endpoint = sfo2.digitaloceanspaces.com
region = sfo2
acl = public
access_key_id = *****
secret_access_key = *****

[digitalocean2]
type=s3
env_auth = false
endpoint = sfo2.digitaloceanspaces.com
region = sfo2
acl = public
access_key_id = *****
secret_access_key = *****

thestigma · October 15, 2019, 4:08am

error 400 usually indicates that you are trying to perform an illegal operation - the server is unable to process your request. So usually this means something is "outside of specs" so to speak. Permission issues should normally give a different error code. A bug in the backend code would also not be impossible - but that's unlikely unless there was a recent big change in the API or something.

For example if the bucket does not exist or you are not giving the correct syntax to access the bucket - that could be the cause.

digitalocean:sc-images/test_bucket
In this, the bucket would be sc-images
Is that what you intended? Because your test_bucket would be a folder in the bucket, not a bucket itself.

Aside from that basic analysis of what the error code indicates I am not sure how to advice you on this.
While you wait for others to come assist it always helps to provide:

a DEBUG log of the problem as it happens (append -vv or --log-level DEBUG to your command)
The contents of your rclone.conf file, so we can see your setup and verify it is correct.

Please note - there may be sensitive information in the config file. Please redact any passwords, secrets or crypt-keys you might have before posting.

Macario · October 15, 2019, 4:27am

In this, the bucket would be sc-images
Is that what you intended? Because your test_bucket would be a folder in the bucket, not a bucket itself

Yes, that's indeed what i intended, i actually successfully manged to copy files from it, it's just when trying to copy file to it that it gives me that error.

ncw · October 15, 2019, 9:15am

The problem is this - if you remove acl - public it will work.

It looks like public isn't one of the ACLs digital ocean support.

system · October 18, 2019, 9:15am

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.