Hello, I have a use case for rclone mount but for security reasons my setup doesn't allow me to store the storage credentials (S3 secret key in this case) on the machine where I'm dispatching the rclone mount command. As a workaround I'd like to stand up an independent proxy server (where storing the credentials is OK) in front of the client that would intercept the outgoing network request(s) to S3 and inject the credentials prior to forwarding the requests to S3. I see in the rclone FAQ that proxies are supported, would my use case be covered here? I'm specifically worried that the traffic to S3 would be encrypted before the traffic hit my proxy server in which case I wouldn't be able to inject the credentials, but I could see other issues coming up as well. Thanks!
I think you would find this quite hard to make work properly as you'd need to remake the AWS signatures. Not impossible but quite hard.
Perhaps easier might be to export the s3 bucket using rclone serve webdav s3:mybucket on the proxy server and mount that export on the maching which can't have the credentials. You could use sftp in place of webdav which may work better if you aren't limited to http based protocols.
@ncw thanks for jumping in here! so to confirm, rclone initiates the https connection prior to hitting the proxy, such that we would need to terminate the https connection (ie remake the AWS signatures) at the proxy and then initiate a new https connection with AWS after we've injected the credentials?
Yes. However you could set a non http endpoint in the s3 config to make your life a little easier. This would stop you having to do proxy config as rclone would go straight to your proxy as that is how it would be configured.
Typically when proxying https connections the client will make a connection to the proxy (typically using HTTP) and send the command CONNECT remote_server:443 HTTP/1.1 (or similar). The proxy will make the connect and respond back with a HTTP/1.1 200 response and then the client is basically talking "directly" to the remote server and will start the TLS stuff as normal.
These proxies can't see or modify the data; it's still all protected by the TLS encryption.
Now there are MITM ("man in the middle") proxy servers. These pretend to be the remote server by faking a certificate (they have a trusted CA certificate you add to your cert store, so the client trusts it). This proxy then makes a connection to the real server and pretends to be the client. Some corporations use this for "data loss prevention" because they now get to see the data.
In theory you could write custom code for a MITM proxy to modify S3 traffic to inject the key and calculate the HMAC signatures based on the secret. I don't know of any that can do this out of the box, though!
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.