Rclone can not copy from KS3 to KS3

Help and Support
1

What is the problem you are having with rclone?

rclone can upload local file to ks3, and download file from ks3, but can't copy file from ks3 to ks3

What is your rclone version (output from rclone version)

on master branch build
rclone v1.54.0-DEV
- os/arch: darwin/amd64
- go version: go1.15.3

Which OS you are using and how many bits (eg Windows 7, 64 bit)

macOS

Which cloud storage system are you using? (eg Google Drive)

KS3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

{16:39}~/Desktop/rclone ➭ ./go_build_rclone_go lsf ksyun-online:
testbucket/
testbucket2/
{16:39}~/Desktop/rclone ➭ ./go_build_rclone_go copy IMG_0018.JPG ksyun-online:testbucket/ -P
Transferred:   	   91.420k / 91.420 kBytes, 100%, 212.564 kBytes/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:         1.4s
{16:39}~/Desktop/rclone ➭ ./go_build_rclone_go copy ksyun-online:testbucket/IMG_0018.JPG . -P
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks:                 1 / 1, 100%
Elapsed time:         0.3s
{16:40}~/Desktop/rclone ➭ ./go_build_rclone_go copy ksyun-online:testbucket/ ksyun-online:testbucket2/ -P
2020-11-06 16:40:35 ERROR : IMG_0017.JPG: Failed to copy: InvalidArgument: Invalid Argument
	status code: 400, request id: , host id:
2020-11-06 16:40:35 ERROR : IMG_0018.JPG: Failed to copy: InvalidArgument: Invalid Argument
	status code: 400, request id: , host id:
2020-11-06 16:40:35 ERROR : Attempt 1/3 failed with 2 errors and: InvalidArgument: Invalid Argument
	status code: 400, request id: , host id:
2020-11-06 16:40:35 ERROR : IMG_0017.JPG: Failed to copy: InvalidArgument: Invalid Argument
	status code: 400, request id: , host id:
2020-11-06 16:40:35 ERROR : IMG_0018.JPG: Failed to copy: InvalidArgument: Invalid Argument
	status code: 400, request id: , host id:
2020-11-06 16:40:35 ERROR : Attempt 2/3 failed with 2 errors and: InvalidArgument: Invalid Argument
	status code: 400, request id: , host id:
2020-11-06 16:40:35 ERROR : IMG_0017.JPG: Failed to copy: InvalidArgument: Invalid Argument
	status code: 400, request id: , host id:
2020-11-06 16:40:35 ERROR : IMG_0018.JPG: Failed to copy: InvalidArgument: Invalid Argument
	status code: 400, request id: , host id:
2020-11-06 16:40:35 ERROR : Attempt 3/3 failed with 2 errors and: InvalidArgument: Invalid Argument
	status code: 400, request id: , host id:
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 2 (retrying may help)
Elapsed time:         0.8s
2020/11/06 16:40:35 Failed to copy with 2 errors: last error was: InvalidArgument: Invalid Argument
	status code: 400, request id: , host id: 
{16:40}~/Desktop/rclone ➭

The rclone config contents with secrets removed.

[ksyun-online]
type = s3
provider = Other
access_key_id = 
secret_access_key = 
endpoint = ks3-cn-beijing.ksyun.com
region = BEIJING

A log from the command with the -vv flag

2020-11-06 16:43:07 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2020-11-06 16:43:07 DEBUG : S3 bucket testbucket2: Waiting for checks to finish
2020-11-06 16:43:07 DEBUG : S3 bucket testbucket2: Waiting for transfers to finish
2020-11-06 16:43:07 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2020-11-06 16:43:07 DEBUG : HTTP REQUEST (req 0xc000190800)
2020-11-06 16:43:07 DEBUG : PUT /testbucket2/IMG_0017.JPG HTTP/1.1
Host: ks3-cn-beijing.ksyun.com
User-Agent: rclone/v1.54.0-DEV
Content-Length: 0
Authorization: XXXX
X-Amz-Acl: private
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Copy-Source: testbucket/IMG_0017.JPG
X-Amz-Date: 20201106T084307Z
X-Amz-Metadata-Directive: COPY
Accept-Encoding: gzip
2020-11-06 16:43:07 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2020-11-06 16:43:07 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2020-11-06 16:43:07 DEBUG : HTTP REQUEST (req 0xc000271400)
2020-11-06 16:43:07 DEBUG : PUT /testbucket2/IMG_0018.JPG HTTP/1.1
Host: ks3-cn-beijing.ksyun.com
User-Agent: rclone/v1.54.0-DEV
Content-Length: 0
Authorization: XXXX
X-Amz-Acl: private
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Copy-Source: testbucket/IMG_0018.JPG
X-Amz-Date: 20201106T084307Z
X-Amz-Metadata-Directive: COPY
Accept-Encoding: gzip
2020-11-06 16:43:07 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2020-11-06 16:43:07 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2020-11-06 16:43:07 DEBUG : HTTP RESPONSE (req 0xc000190800)
2020-11-06 16:43:07 DEBUG : HTTP/1.1 400 Bad Request
Transfer-Encoding: chunked
Connection: keep-alive
Content-Type: application/xml
Date: Fri, 06 Nov 2020 08:43:07 GMT
Server: Tengine
X-Application-Context: application
X-Kss-Request-Id: 0a80b9dffc4945adbf8cc92e7d5b369b

ea
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><Error><Code>InvalidArgument</Code><Message>Invalid Argument</Message><Resource>/testbucket2/IMG_0017.JPG</Resource><RequestId>0a80b9dffc4945adbf8cc92e7d5b369b</RequestId></Error>
0
2020-11-06 16:43:07 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2020-11-06 16:43:07 ERROR : IMG_0017.JPG: Failed to copy: InvalidArgument: Invalid Argument
	status code: 400, request id: , host id:
2020-11-06 16:43:07 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2020-11-06 16:43:07 DEBUG : HTTP RESPONSE (req 0xc000271400)
2020-11-06 16:43:07 DEBUG : HTTP/1.1 400 Bad Request
Transfer-Encoding: chunked
Connection: keep-alive
Content-Type: application/xml
Date: Fri, 06 Nov 2020 08:43:07 GMT
Server: Tengine
X-Application-Context: application
X-Kss-Request-Id: c0684f4fde4641d38cb4aaa9e70d02fc

ea
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><Error><Code>InvalidArgument</Code><Message>Invalid Argument</Message><Resource>/testbucket2/IMG_0018.JPG</Resource><RequestId>c0684f4fde4641d38cb4aaa9e70d02fc</RequestId></Error>
0
2020-11-06 16:43:07 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2

instead of using a beta, have you tried the latest stable from the rclone website?

can you add -vv to your commands, for debug output and post that log?

3

This request looks OK

But the result isn't very helpful - it doesn't give any clues.

I haven't been able to find server side copy in their documentation - maybe it isn't supported. Can you find it in their docs?

I've been trying to sign up for a KS3 account but I'm stuck at the identity stage - they don't think my passport is a passport!

4

Hello, I tried the latest version of rclone, the problem still exists。
The result of adding the -vv parameter is given above

5

Hello, Thank you for your reply, Server Side Copy, KS3 provides this interface, you can see if it can be used，https://endocs.ksyun.com/documents/27926，If necessary, I can give you my KingSoft test account

6

Thanks for finding the doc. That appears to use a non-standard x-kss-copy-source rather than x-amz-copy-source. Though the other methods which do work appear to work with x-amz-* so maybe that isn't the problem.

That Copy routine works with a lot of other S3 compatible providers so I guess it is some incompatibility with KS3.

I have managed to send my passport off now so hopefully I'll have my own KS3 account shortly to test! I can ask tech support for help too hopefully.

7

OK，I checked the object storage API documents of other cloud vendors. Tencent COS is x-cos-copy-source and Huaweicloud OBS is x-obs-copy-source, so it may indeed be incompatible with KS3.

8

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.