Can't use sse_customer_key with non-printable charactersin config file

Feature
1

The feature request page says that I need to make a forum post in order to submit a report, so I'll start here :wink:

I'm wanting to clone a backup from one bucket to another -- shouldn't be too hard with rclone, except that the SSE-C key I'm using does not seem to be usable in the config file because it is not 32 printable characters, so there seems to be no way to put it into the file.

I was able to make it work from the command line using this:

rclone copy testfile rbackup:/backup-bucket/backup2/ --s3-sse-customer-key="base64 -d < <(echo 'l/83cl9skdoczIu+ZmqHbYX5SR8lCQk6jsKg35bmeVg=')" --s3-sse-customer-algorithm=AES256

That's just an arbitrary key I created using openssl rand -base64 32 but I can't find a way to put that in the config file.

I had two thoughts on how this could work:

  1. It could allow you to provide the key base64 encoded -- either by detecting "this is too long, must be base64 encoded" or allowing a prefix (e.g. base64:)
  2. It could allow you to provide an obscured value, since it seems I can pass the output of the above like base64 -d < <(echo 'l/83cl9skdoczIu+ZmqHbYX5SR8lCQk6jsKg35bmeVg=') | rclone obscure -

It actually seems like the customer-key should be obscured anyway -- not that most keys you are likely to be able to read, of course...

Any thoughts? Is there a solution I'm missing? Limiting the AES256 keys to printable characters inherently limits the security of the key anyway, which seems less than ideal...

2

Hi Richard,

I am not into the details nor do I have S3, but did you try something like

rclone config update yourS3remote:  sse_customer_algorithm AES256  sse_customer_key "base64 -d < <(echo 'l/83cl9skdoczIu+ZmqHbYX5SR8lCQk6jsKg35bmeVg=')"

Documentation: rclone config update

3

Good thought! I tried it and got Error: found key without value

EDIT: And then I realized I was dumb and missed the quotes, so I tried it again and it looks like it worked. It's ... definitely not ideal still, but it may be usable. We'll see what happens when I try an actual pre-determined key.

4

Very interesting... it does actually work! You need the quotes and to use the correct syntax, but this works:

rclone config update yours3remote sse_customer_algorithm AES256 sse_customer_key "$(base64 -d < <(echo 'base64-encoded-string-here'))"

You need all of the quotes, parens, dollar sign, etc all exactly as they are, but that works at least in zsh and should in bash.

I still think this should a) be obfuscated in the config file and b) support something user editable, but it's a usable workaround!

5

Good catch!

a) You could also encrypt the entire config: https://rclone.org/docs/#configuration-encryption

b) I will pass the decision on something user editable to @ncw when he is back. He seems to be taking a well-earned break at the moment.

Glad we found a usable workaround!

6

As a developer myself I certainly do not begrudge someone the chance to take a break =] I'm just trying to help make an already fantastic tool a little bit better and more usable.

Hopefully this thread will at least help the next person who needs to do this!

I did file an issue for it on github: Obscure or allow obscuring sse_customer_key · Issue #6400 · rclone/rclone · GitHub

1 Like
7

I'm back now from summer break! I'll take this up further on the issue.

8

I hope it was a good break =]

1 Like
9

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.