Presigned Keys In Rclone Serve S3

What is the problem you are having with rclone?

I am running rclone serve s3, and I am getting the error UnsupportedAlgorithmEncountered an unsupported algorithm. which I have no idea how to fix. The logs seem to suggest that access being denied, but I am providing URLs with Presigned keys, which should bypass needing authentication. Is this something that is not supported in Rclone? That would be a huge bummer, and something that really should be looked into.

Run the command 'rclone version' and share the full output of the command.

rclone v1.65.2
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.4.0-33-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.6
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

WebDav

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone serve s3 union_remote: --addr 0.0.0.0:9090 --auth-key \"$access_key,$secret_key\"

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[union_remote]
type = union
upstreams = webdav0:/ webdav1:/ webdav2:/ webdav3:/ webdav4:/

[webdav0]
type = webdav
url = http://100.97.90.47:8080
vendor = other
user = XXX
pass = XXX

[webdav1]
type = webdav
url = http://100.105.58.84:8080
vendor = other
user = XXX
pass = XXX

[webdav2]
type = webdav
pass = XXX
url = http://100.68.225.25:8080
vendor = other
user = XXX

[webdav3]
type = webdav
pass = XXX
url = http://100.78.25.28:8080
vendor = other
user = XXX

[webdav4]
type = webdav
url = http://100.101.237.70:8080
vendor = other
user = XXX
pass = XXX

A log from the command that you were trying to run with the -vv flag

Feb 05 04:50:43 [hostname_redacted] rclone[77835]: INFO  : serve s3: Access Denied: 100.122.89.71:37342 => /[cache_redacted]/32609bdc57ce41eb61ea39b44203516af9b3fb76/%!B(MISSING)MagicStar%!D(MISSING)%!C(MISSING)huubou%!n(MISSING)o%!A(MISSING)lice%!-(MISSING)%!M(MISSING)inna%!d(MISSING)e%!T(MISSING)sukurou%!K(MISSING)agaku%!d(MISSING)e%!K(MISSING)andan%!R(MISSING)ecipe%!E(MISSING)P03%!E(MISSING)ND%5BWEBDL%!D(MISSING)%5B1080p%!D(MISSING)/%!B(MISSING)MagicStar%!D(MISSING)%!C(MISSING)huubou%!n(MISSING)o%!A(MISSING)lice%!-(MISSING)%!M(MISSING)inna%!d(MISSING)e%!T(MISSING)sukurou%!K(MISSING)agaku%!d(MISSING)e%!K(MISSING)andan%!R(MISSING)ecipe%!E(MISSING)P03%!E(MISSING)ND%5BWEBDL%!D(MISSING)%5B1080p%!D(MISSING).mkv?AWSAccessKeyId=vHNhZ9FZK124XfCycpU2sDzJyjoWKgWyUJwme0UDfSQAJdSDCwiwQpM0lduUqsh&Signature=LOP3Tw0eeco9rtlPJ1fKXHpkO6A%!D(MISSING)&Expires=1707111337
Feb 05 04:50:45 [hostname_redacted] rclone[77835]: INFO  : serve s3: Access Denied: 100.122.89.71:37344 => /[cache_redacted]/32609bdc57ce41eb61ea39b44203516af9b3fb76/%!B(MISSING)MagicStar%!D(MISSING)%!C(MISSING)huubou%!n(MISSING)o%!A(MISSING)lice%!-(MISSING)%!M(MISSING)inna%!d(MISSING)e%!T(MISSING)sukurou%!K(MISSING)agaku%!d(MISSING)e%!K(MISSING)andan%!R(MISSING)ecipe%!E(MISSING)P03%!E(MISSING)ND%5BWEBDL%!D(MISSING)%5B1080p%!D(MISSING)/%!B(MISSING)MagicStar%!D(MISSING)%!C(MISSING)huubou%!n(MISSING)o%!A(MISSING)lice%!-(MISSING)%!M(MISSING)inna%!d(MISSING)e%!T(MISSING)sukurou%!K(MISSING)agaku%!d(MISSING)e%!K(MISSING)andan%!R(MISSING)ecipe%!E(MISSING)P03%!E(MISSING)ND%5BWEBDL%!D(MISSING)%5B1080p%!D(MISSING).mkv?AWSAccessKeyId=vHNhZ9FZK124XfCycpU2sDzJyjoWKgWyUJwme0UDfSQAJdSDCwiwQpM0lduUqsh&Signature=LOP3Tw0eeco9rtlPJ1fKXHpkO6A%!D(MISSING)&Expires=1707111337
Feb 05 04:50:58 [hostname_redacted] rclone[77835]: INFO  : serve s3: Access Denied: 100.83.53.41:59942 => /[cache_redacted]/32609bdc57ce41eb61ea39b44203516af9b3fb76/[MagicStar]%!C(MISSING)huubou%!n(MISSING)o%!A(MISSING)lice%!-(MISSING)%!M(MISSING)inna%!d(MISSING)e%!T(MISSING)sukurou%!K(MISSING)agaku%!d(MISSING)e%!K(MISS

welcome to the forum,

keep in mind, the rclone serve s3 was just added to rclone and is experimental.

i did a quick test, just followed the rclone docs and worked for me

rclone.exe serve s3 d:\files\test -vv --auth-key=ACCESS_KEY_ID,SECRET_ACCESS_KEY

and here the exact rclone remote used to access the s3 server, as per rclone doc example

[serves3]
type = s3
provider = Rclone
endpoint = http://127.0.0.1:8080/
access_key_id = ACCESS_KEY_ID
secret_access_key = SECRET_ACCESS_KEY
use_multipart_uploads = false

and here exact rclone copy command

rclone copy file.ext serves3:bucket -vv
2024/02/05 09:04:02 DEBUG : rclone: Version "v1.65.2" starting with parameters ["rclone" "copy" "file.ext" "serves3:bucket" "-vv"]
2024/02/05 09:04:02 DEBUG : Creating backend with remote "file.ext"
2024/02/05 09:04:02 DEBUG : Using config file from "c:\\data\\rclone\\rclone.conf"
2024/02/05 09:04:02 DEBUG : fs cache: adding new entry for parent of "file.ext", "//?/c:/data/rclone"
2024/02/05 09:04:02 DEBUG : Creating backend with remote "serves3:bucket"
2024/02/05 09:04:02 DEBUG : Resolving service "s3" region "us-east-1"
2024/02/05 09:04:02 DEBUG : S3 bucket bucket: Disabling multipart uploads
2024/02/05 09:04:02 DEBUG : file.ext: Need to transfer - File not found at Destination
2024/02/05 09:04:02 INFO  : S3 bucket bucket: Bucket "bucket" created with ACL ""
2024/02/05 09:04:02 DEBUG : file.ext: md5 = ae0913fffdea4b03e1290c842be67924 OK
2024/02/05 09:04:02 INFO  : file.ext: Copied (new)
2024/02/05 09:04:02 INFO  :
Transferred:             14 B / 14 B, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%

Yes, the s3 works completely fine when used with Rclone itself, but did you even read the post title? I am dealing with a different problem entirely. I understand that the serve s3 is experimental, which is why I want to improve it, by making issues known.

I don't know the answer to whether rclone serve S3 works with presigned requests, but from your log it looks like it doesn't.

It should be relatively easy to implement presigned requests though as it is just passing the headers as URL parameters...

I suggest you make an rclone issue to discuss further.

1 Like

Thank you, made an issue here -> serve s3: Allow Presigned Requests for Serving S3 · Issue #7616 · rclone/rclone · GitHub

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.