Rclone 1.53.1 can not list buckets on KingSoft Cloud S3 Service

What is the problem you are having with rclone?

rclone 1.53.1 can not run command "lsf" on ksyun

What is your rclone version (output from rclone version)

rclone v1.53.1-DEV

Which OS you are using and how many bits (eg Windows 7, 64 bit)

MacOS 10.15.7

Which cloud storage system are you using? (eg Google Drive)

Kingsoft Cloud

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone lsf -vv ksyun-online:

The rclone config contents with secrets removed.

[ksyun-online]
type = s3
provider = Other
access_key_id = 
secret_access_key = 
endpoint = ks3-cn-beijing.ksyun.com

A log from the command with the -vv flag

2020/10/25 17:32:42 DEBUG : rclone: Version "v1.53.1-DEV" starting with parameters ["rclone" "lsf" "-vv" "--dump" "headers" "--dump" "bodies" "ksyun-online:"]
2020/10/25 17:32:42 DEBUG : Using config file from "/Users/honest1y/.config/rclone/rclone.conf"
2020/10/25 17:32:42 DEBUG : Creating backend with remote "ksyun-online:"
2020/10/25 17:32:42 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2020/10/25 17:32:42 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2020/10/25 17:32:42 DEBUG : HTTP REQUEST (req 0xc00021a300)
2020/10/25 17:32:42 DEBUG : GET / HTTP/1.1
Host: ks3-cn-beijing.ksyun.com
User-Agent: rclone/v1.53.1-DEV
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c495991b7852b855
X-Amz-Date: 20201025T093242Z
Accept-Encoding: gzip

2020/10/25 17:32:42 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2020/10/25 17:32:42 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2020/10/25 17:32:42 DEBUG : HTTP RESPONSE (req 0xc00021a300)
2020/10/25 17:32:42 DEBUG : HTTP/1.1 400 Bad Request
Transfer-Encoding: chunked
Connection: keep-alive
Content-Type: application/xml; charset=UTF-8
Date: Sun, 25 Oct 2020 09:32:42 GMT
Server: Tengine
X-Amz-Request-Id: d9315405bca54009ad443b560945552b
X-Application-Context: application
X-Kss-Request-Id: d9315405bca54009ad443b560945552b

e9
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<Error>
    <Code>InvalidArgument</Code>
    <Message>Invalid Argument</Message>
    <Resource>/</Resource>
    <RequestId>d9315405bca54009ad443b560945552b</RequestId>
</Error>

0

2020/10/25 17:32:42 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2020/10/25 17:32:42 ERROR : : error listing: InvalidArgument: Invalid Argument
	status code: 400, request id: d9315405bca54009ad443b560945552b, host id: 
2020/10/25 17:32:42 DEBUG : 6 go routines active
2020/10/25 17:32:42 Failed to lsf with 2 errors: last error was: error in ListJSON: InvalidArgument: Invalid Argument
	status code: 400, request id: d9315405bca54009ad443b560945552b, host id: 

I got part of the way through integrating KS3

Can you try this binary

v1.54.0-beta.4841.2f0ef2e98.fix-ks3 on branch fix-ks3 (uploaded in 15-30 mins)

And your config should look something like this

[ks3]
type = s3
provider = Other
env_auth = false
access_key_id = XXX
secret_access_key = XXX
endpoint = ks3-cn-guangzhou.ksyun.com
force_path_style = false
v2_auth = true

Thank you so much, I used the branch of fix-ks3, the test was successful, thank you again :yum:

I use rclone v1.53.1-DEV, ang try to add config "region = BEIJING" ,It also works.

Great!

Can you show me what your config looks like (remove the secrets :slight_smile: please?

Are you using v2_auth = true?

Which version is this? Is it the KS3 branch?

No problem!
rclone version

{20:12}~ ➭ rclone version
rclone v1.53.1-DEV
- os/arch: darwin/amd64
- go version: go1.15.2

config looks like

[ksyun-online]
type = s3
provider = Other
region = BEIJING
access_key_id = 
secret_access_key = 
endpoint = ks3-cn-beijing.ksyun.com

I did not use v2_auth = true

That looks like one you compiled yourself.

Is it from the master branch or the fix-ks3 branch? I can't see from v1.53.1-DEV

It's from the master branch

[root@i-papkjzwm ~]# rclone version
rclone v1.52.2
- os/arch: linux/amd64
- go version: go1.14.4
[root@i-papkjzwm ~]# cat .config/rclone/rclone.conf
[ksyun-online]
type = s3
provider = Other
access_key_id = 
secret_access_key = 
endpoint = ks3-cn-beijing.ksyun.com
region = BEIJING
[root@i-papkjzwm ~]# rclone lsf ksyun-online:
testbucket/
[root@i-papkjzwm ~]# 

So it looks like my KS3 patches are no longer needed?

maybe, but I am using the code of the fix-ks3 branch, It's no need to configure the region parameter :yum:

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.