August 27, 2022, 7:13pm
The feature request page says that I need to make a forum post in order to submit a report, so I'll start here
I'm wanting to clone a backup from one bucket to another -- shouldn't be too hard with rclone, except that the SSE-C key I'm using does not seem to be usable in the config file because it is not 32 printable characters, so there seems to be no way to put it into the file.
I was able to make it work from the command line using this:
rclone copy testfile rbackup:/backup-bucket/backup2/ --s3-sse-customer-key="base64 -d < <(echo 'l/83cl9skdoczIu+ZmqHbYX5SR8lCQk6jsKg35bmeVg=')
That's just an arbitrary key I created using
openssl rand -base64 32 but I can't find a way to put that in the config file.
I had two thoughts on how this could work:
It could allow you to provide the key base64 encoded -- either by detecting "this is too long, must be base64 encoded" or allowing a prefix (e.g. base64:)
It could allow you to provide an obscured value, since it seems I can pass the output of the above like
base64 -d < <(echo 'l/83cl9skdoczIu+ZmqHbYX5SR8lCQk6jsKg35bmeVg=') | rclone obscure -
It actually seems like the customer-key should be obscured anyway -- not that most keys you are likely to be able to read, of course...
Any thoughts? Is there a solution I'm missing? Limiting the AES256 keys to printable characters inherently limits the security of the key anyway, which seems less than ideal...
August 28, 2022, 2:17pm
I am not into the details nor do I have S3, but did you try something like
rclone config update yourS3remote: sse_customer_algorithm AES256 sse_customer_key "base64 -d < <(echo 'l/83cl9skdoczIu+ZmqHbYX5SR8lCQk6jsKg35bmeVg=')"
rclone config update
August 29, 2022, 9:58pm
Good thought! I tried it and got
Error: found key without value
EDIT: And then I realized I was dumb and missed the quotes, so I tried it again and it looks like it worked. It's ... definitely not ideal still, but it may be usable. We'll see what happens when I try an actual pre-determined key.
August 29, 2022, 10:08pm
Very interesting... it does actually work! You need the quotes and to use the correct syntax, but this works:
rclone config update yours3remote sse_customer_algorithm AES256 sse_customer_key "$(base64 -d < <(echo 'base64-encoded-string-here'))"
You need all of the quotes, parens, dollar sign, etc all exactly as they are, but that works at least in zsh and should in bash.
I still think this should a) be obfuscated in the config file and b) support something user editable, but it's a usable workaround!
August 30, 2022, 6:54am
a) You could also encrypt the entire config:
b) I will pass the decision on something user editable to
@ncw when he is back. He seems to be taking a well-earned break at the moment.
Glad we found a usable workaround!
August 30, 2022, 4:04pm
As a developer myself I certainly do not begrudge someone the chance to take a break =] I'm just trying to help make an already fantastic tool a little bit better and more usable.
Hopefully this thread will at least help the next person who needs to do this!
I did file an issue for it on github:
Obscure or allow obscuring sse_customer_key · Issue #6400 · rclone/rclone · GitHub
September 6, 2022, 10:08am
I'm back now from summer break! I'll take this up further on the issue.
September 6, 2022, 3:03pm
I hope it was a good break =]
November 5, 2022, 3:03pm
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.