AWS S3 SSE-C - how to generate keys?

Off Topic
1

i am having a nightmare trying to create values for
--s3-sse-customer-key
and
--s3-sse-customer-key-md5

i found python code that many claim should work.
but when i use the it, i get the following error in rclone log.

2020/11/11 18:33:04 DEBUG : rclone: Version "v1.53.2" starting with parameters ["c:\\data\\rclone\\scripts\\rclone.exe" "copy" "c:\\ex\\wg" "wasabi01:sssasdffdsa" "--s3-sse-customer-algorithm=AES256" "--s3-sse-customer-key=bbb6434b1d1e3eb63fabd2491625f5fd" "--s3-sse-customer-key-md5=b1b2ecbc13576b716ad92526590539cb" "--log-file=log.sse.txt" "--log-level=DEBUG"]
2020/11/11 18:33:04 ERROR : wg.md5: Failed to copy: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidArgument</Code><Message>The calculated MD5 hash of the key did not match the hash that was provided.</Message><ArgumentName>x-amz-server-side-encryption</ArgumentName><ArgumentValue>null</ArgumentValue><RequestId>FCE77A20651195DB</RequestId><HostId>eRUKTQhhoctH0Zi5KzWeIN5dmUI1X/WeGywqxxON2PCb8PxDXP0Y+spP8gzdug7Z6UByEaxrIEco</HostId></Error>

anybody have python code, go code or a website that can generate the correct values
thanks

2

Nobody answered this and it's been years, so I'll add an answer in hopes it saves someone a bit of time:

  • You don't need to provide "--s3-sse-customer-key-md5" because rclone will calculate it for you
  • You do need to make sure "--sse-customer-key" is something that can be passed via command line; I like to use openssl to generate a random 32 byte (256 bit) key but when I tried that it didn't work. I haven't dug around to see if there is another way to pass it in; I was hoping it would accept a base64 encoded key, but no such luck. I finally just used lastpass to make a 32 character password with all character groups, and that's probably good enough realistically.
  • You also need to provide "--s3-sse-customer-algorithm=AES256" -- it is possible that some s3 compatible providers support something other than AES256, but s3 itself and most others I've seen only support AES256, which is too bad but is what it is.

You can optionally set this in the confguration file, for example:

[rbackup]
type = s3
provider = ceph
access_key_id = (redacted)
secret_access_key = (redacted)
endpoint = https://s3-files.mycompanydomain.com
sse_customer_algorithm = AES256
sse_customer_key = M5V*7pS9YYxtQ6Vfg9mPurpqjbtvWysy
1 Like
3

... and since I'm apparently incapable of leaving well enough alone, here is a way to do it using unprintable characters from a base64 encoded string, at least with bash or zsh:

rclone copy . rbackup:/bucket-name/enc-backup/ --s3-sse-customer-key="base64 -d < <(echo 'l/83cl9skdoczIu+ZmqHbYX5SR8lCQk6jsKg35bmeVg=')`" --s3-sse-customer-algorithm=AES256``

that key prints to bash as ��7r_l��̋�fj�m��I% :� ߖ�yX =]

4

hello and welcome to the forum,

to generate the md5 in python

password = b'12345678901234567890123456789012'
encryption_key_md5 = base64.b64encode(hashlib.md5(password).digest())
print(encryption_key_md5)

b'dnF5x6K/8ZZRzpfSlMMM+w=='

and the remote would look like

[wasabi_ssectest_remote]
type = s3
provider = Wasabi
access_key_id = redacted
secret_access_key = redacted
endpoint = s3.us-east-2.wasabisys.com
sse_customer_algorithm = AES256
sse_customer_key = 12345678901234567890123456789012
sse_customer_key_md5 = dnF5x6K/8ZZRzpfSlMMM+w==
1 Like