It is hoped that rclone can be compatible with KS3's signature algorithm

Help and Support
1

What is the problem you are having with rclone?

Background: when using rclone to upload a file larger than 5G to Kingsoft cloud (KS3), an error will be reported: signing error. This is because when users upload files to KS3 in blocks, the Etag calculation method of KS3 is different from that of S3, while rclone only supports the signature algorithm of S3, not KS3.
Solution: it is hoped that rclone can be compatible with KS3's Etag algorithm, and KS3 can provide code for official review by rclone.

What is your rclone version (output from rclone version)

The latest version

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Which cloud storage system are you using? (eg Google Drive)

Kingsoft cloud KS3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

.rclone copyto/root/ExchangeServer2016-x64-cu11.iso ks3-sh:fzy-shtest-181024/ExchangeServer.iso

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

2

Looks like it's just a feature request to support the new signing for that backend.

You can just submit a feature request here:

And just share any code/information in the feature request.

3

If you'd like to provide a pull request that would be perfect.

You'd need to make a new provider in the s3 backend code. This would give a chance to set endpoints etc. It would also give a chance to set new signer code.

4

Thank you. There's another question. Which branch should we add our code to?

5

You should base your pull request off rclone's master branch. What you call your own branch is up to you :slight_smile:

6

I'm sorry that I didn't describe my problem clearly. Our R&D personnel are not familiar with the code of rclone. I want to ask which module is the code of MD5 algorithm of rclone? :sweat_smile:

1587873034(1)
1587873034(1)1206×620 38.7 KB

7

For a code overview see here: https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#code-organisation

Rclone uses go's standard library MD5 routines.

8

Hello, Nick!
our R & D personnel found that when using rclone to upload large files to KS3, as long as you add a parameter “--ignore-checksum”, you can solve the problem of upload failure. So I want to ask if you can add the configuration instructions of KS3 to rclone official website（https://rclone.org/）， just like the configuration examples of other cloud service providers, so it is more convenient for users to use.

9

This happens for files bigger than 5 GB? Is that correct?

What I need for that to happen is a an account I can run tests against. I would run the daily integration tests against it too.

Does KS3 have a free tier or could you get me a free account?

10

Yes, files larger than 5g. We can provide you a free test account, I can send it to your email.Can you give me an email address?

11

My email is nick@craig-wood.com . Ideally can you give me an account I can use permanently that I can add into the integration tests? This will ensure rclone remains compatible with KS3. The integration tests use < 1GB of data and a 100MB of transfer or so a day.

12

The account we provide for you is long-term effective. You can contact us if you encounter any use problems. If the account needs to be deactivated, we will inform you in advance and provide a new account. Do you think it is OK? :slightly_smiling_face:

13

Sounds great - thank you!

14

Account information has been sent to you, thank you for your help! :smiley:

1 Like
15

I have received it thank you and I've replied by email.

16

Hello, Nick,
I have solved the problems mentioned in your email. I sent the new configuration file to your email. You can try it. :smiley:

17

Apologies for the delay! I've replied to your email. It is working a bit now ...