Rclone serve s3 <remote>


#1

@ncw what are your thoughts on a feature like serving a remote over the s3 protocol?

Use cases:

  • Rclone as s3 proxy layer for other cloud storage backends.
  • Quick s3 endpoint over a filesystem (local remote) for development purposes.

I know this feature is something similar to what Minio does (basically an s3-like server), but I’m thinking of this as a lightweight solution that supports many more remote backends.

Initially, the protocol implementation could be limited to just the most common operations, so that this feature is not a monumental undertaking. Credentials could be specified inline, and the region could just be the default us-east-1.


#2

It is a nice idea. It might be that minio has some code rclone can use, I’m not sure.

The S3 protocol is large and complicated!

Do you want to work on this?


#3

Yes, I would love to work on this!


#4

:smile:

I found this go s3 project which looks like it might be easier to integrate than minio


#5

That project is in Java, so I think it might not be easier to integrate with :smile:

Perhaps, are you referring to this one?


#6

Yes that is right - sorry copy pasted the wrong URL!