KMS for S3 file creation

This might be a weird question, but how do I use KMS with rclone? I moved up to preprod and didn’t know our instance uses KMS alongside IAM roles, couldn’t see a way to use it with rclone.

Found some more info, apologies, I’m new to AWS.
This is what’s stoppin’ me from writing to the s3

https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html

{
“Version”:“2012-10-17”,
“Id”:“PutObjPolicy”,
“Statement”:[{
“Sid”:“DenyUnEncryptedObjectUploads”,
“Effect”:“Deny”,
“Principal”:"",
“Action”:“s3:PutObject”,
“Resource”:"arn:aws:s3:::YourBucket/
",
“Condition”:{
“StringNotEquals”:{
“s3:x-amz-server-side-encryption”:“aws:kms”
}
}
}
]
}

Last bump. It looks like I need the rclone version of --server-side-encryption aws:kms and --ssekms-key-id

There is an open issue about KMS and rclone here: https://github.com/ncw/rclone/issues/1824 which will cause you a problem eventually…

rclone doesn’t support this at the moment - if you’d like it to then please make a new issue on github with lots of info.

I don’t really understand what needs to be done so links into the sdk would be super helpful!

AWS recently provided a way to set the default encryption behavior for an S3 bucket. I have not tried to re-create your issue to test this idea, but I think setting the default encryption behavior may provide a work around for not yet being able to see the required S3 KMS flags in rclone yet.
https://docs.aws.amazon.com/AmazonS3/latest/dev/bucket-encryption.html

I’m not 100% sure, but a quick look at the SDK looks like the s3crypto service may contain what’s needed to implement this: https://docs.aws.amazon.com/sdk-for-go/api/service/s3/s3crypto/

I’m new on the forum so I couldn’t get all the links I wanted to reference in a single post. I also wanted to mention good general AWS documentation about S3 and KMS encryption is available at:

I had one more link. For it, just search for “How to Prevent Uploads of Unencrypted Objects to Amazon S3” on the AWS blog.

Nice links @mvance - thanks. Can those go in the issue too?

Yes, of course. I didn’t see an issue for this in GitHub yet so I didn’t go ahead and add one. @ncw, let me know if I over looked the issue. If not, @Confirm4Crit would you like to create one or would you like me to? If you do, feel free to include my links or reply back on here so I know when the issue is ready so I can add the links to it.

Since you are interested in this feature too, then why don’t you create the issue and @Confirm4Crit can add anything they feel is relevant to it.

I’ve logged an issue. I did not have enough info to answer all the standard issue questions (version info, platform, command in use, etc.) because I have not started using rclone yet other than to perform a basic test and did have a good S3 bucket with the proper settings to re-create this issue.

I’m interested this though and want to help because this is a feature I see the value in and may want to use in the future. I’m a big fan of restic and the upcoming integration peaked my interest in this project. Currently, I run restic locally and then use the native Google Cloud CLI app to sync my local repo to Google Cloud Storage.

Apologies for not getting to the thread in a bit, just been busy.

It seems like most of my issues have been summed up in the links and what not.
Basically if a policy is on a bucket that requires a KMS key, I don’t have a way to provide it.

In my exact case, I don’t want to generate one or anything, just use one in env auth.
I’m on latest version of rclone, some centos AMI, seems to be influencing any write/put commands, sync, cp etc.
Gets, ls, size, etc, all seem fine.

@mvance thanks for making the issue - it looks fine.

You could use rclone to do this if you wanted too!

rclone already has a server_side_encryption config key - so that can be set to aws:kms.

However there is no way currently of setting this (from aws cli sync docs).

–sse-kms-key-id (string) The AWS KMS key ID that should be used to server-side encrypt the object in S3. Note that you should only provide this parameter if KMS key ID is different the default S3 master KMS key.

Is that a correct summary?

@Confirm4Crit - I’d like to move this discussion to the issue so if you could reply there that would be helpful - thanks.

I forgot - is there a standard way of putting this in the environment?