Is SSE-C supported when using iDrive e2

rgomezc · October 9, 2022, 6:25pm

What is the problem you are having with rclone?

I tried to use the SEE-C encryption when using an iDrive e2 bucket with no luck: when passing the credentials from the config file I always get an AccessDenied, status code 403 error. If I don't pass the credentials, then the transfers fail with md5 hash differ errors.

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.2

os/version: darwin 12.6 (64 bit)
os/kernel: 21.6.0 (arm64)
os/type: darwin
os/arch: arm64
go/version: go1.18.6
go/linking: dynamic
go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

iDrive e2 (S3 compatible)

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone sync

The rclone config contents with secrets removed.

[e2-ba-despues]
type = s3
provider = IDrive
access_key_id = <redacted>
secret_access_key = <redacted>
upload_concurrency = 8
endpoint = v4t2.da.idrivee2-35.com
sse_customer_algorithm = AES256
sse_customer_key = <redacted>

A log from the command with the `-vv` flag

Using the config as pasted here:

2022/10/09 13:03:29 DEBUG : --max-age 99y to 1923-11-03 11:03:29.593123 -0700 MST m=-3122063999.985728707
2022/10/09 13:03:29 DEBUG : rclone: Version "v1.59.2" starting with parameters ["./rclone-arm64" "sync" ".." "e2-ba-despues:/ba-despues-2022/" "--config" "./rclone.config" "-P" "--fast-list" "--max-age" "99y" "--exclude-from" "./exclusiones.txt" "--checkers=24" "--stats=1s" "--log-file=/Users/rodrigo/Dropbox/trabajo/Brujazul/2022/Despues/99_DES_UTILERIAS_NUBE/logs/backup_completo-2022-10-09T130329.log" "-vv" "--ask-password=false"]
2022/10/09 13:03:29 DEBUG : Creating backend with remote ".."
2022/10/09 13:03:29 DEBUG : Using config file from "/Users/rodrigo/Dropbox/trabajo/Brujazul/2022/Despues/99_DES_UTILERIAS_NUBE/rclone.config"
2022/10/09 13:03:29 DEBUG : fs cache: renaming cache item ".." to be canonical "/Users/rodrigo/Dropbox/trabajo/Brujazul/2022/Despues"
2022/10/09 13:03:29 DEBUG : Creating backend with remote "e2-ba-despues:/ba-despues-2022/"
2022/10/09 13:03:29 DEBUG : fs cache: renaming cache item "e2-ba-despues:/ba-despues-2022/" to be canonical "e2-ba-despues:ba-despues-2022"
2022/10/09 13:03:29 DEBUG : .DS_Store: Excluded
2022/10/09 13:03:29 DEBUG : 99_DES_UTILERIAS_NUBE: Excluded
2022/10/09 13:03:29 DEBUG : 01_DES_MATERIAL/.DS_Store: Excluded
2022/10/09 13:03:29 DEBUG : 01_DES_MATERIAL/TEST/.DS_Store: Excluded
2022/10/09 13:03:29 DEBUG : S3 bucket ba-despues-2022: Waiting for checks to finish
2022/10/09 13:03:29 DEBUG : S3 bucket ba-despues-2022: Waiting for transfers to finish
2022/10/09 13:03:29 ERROR : 01_DES_MATERIAL/TEST/2022-09-11 03.30.07.mp4: Failed to copy: AccessDenied: Access Denied.
	status code: 403, request id: 171C78A46F0BCE7D, host id: 
2022/10/09 13:03:30 ERROR : 01_DES_MATERIAL/TEST/2022-09-16 15.07.13.mov: Failed to copy: AccessDenied: Access Denied.
	status code: 403, request id: 171C78A478DB5ED4, host id: 
2022/10/09 13:03:30 ERROR : 01_DES_MATERIAL/TEST/2022-09-19 13.19.03.mp4: Failed to copy: AccessDenied: Access Denied.

Using the config wihout the sse_customer_algorithm and _key (i.e., using the defaults by the provider):

2022/10/09 13:07:27 DEBUG : --max-age 99y to 1923-11-03 11:07:27.559262 -0700 MST m=-3122063999.987137749
2022/10/09 13:07:27 DEBUG : rclone: Version "v1.59.2" starting with parameters ["./rclone-arm64" "sync" "../" "e2-ba-despues:/ba-despues-2022/" "--config" "./rclone.config" "-P" "--fast-list" "--max-age" "99y" "--exclude-from" "./exclusiones.txt" "--checkers=24" "--stats=1s" "--log-file=/Users/rodrigo/Dropbox/Trabajo/Brujazul/2022/Despues/99_DES_UTILERIAS_NUBE/logs/backup_completo-2022-10-09T130727.log" "-vv" "--ask-password=false"]
2022/10/09 13:07:27 DEBUG : Creating backend with remote "../"
2022/10/09 13:07:27 DEBUG : Using config file from "/Users/rodrigo/Dropbox/Trabajo/Brujazul/2022/Despues/99_DES_UTILERIAS_NUBE/rclone.config"
2022/10/09 13:07:27 DEBUG : fs cache: renaming cache item "../" to be canonical "/Users/rodrigo/Dropbox/Trabajo/Brujazul/2022/Despues"
2022/10/09 13:07:27 DEBUG : Creating backend with remote "e2-ba-despues:/ba-despues-2022/"
2022/10/09 13:07:27 DEBUG : fs cache: renaming cache item "e2-ba-despues:/ba-despues-2022/" to be canonical "e2-ba-despues:ba-despues-2022"
2022/10/09 13:07:27 DEBUG : .DS_Store: Excluded
2022/10/09 13:07:27 DEBUG : 99_DES_UTILERIAS_NUBE: Excluded
2022/10/09 13:07:27 DEBUG : 01_DES_MATERIAL/.DS_Store: Excluded
2022/10/09 13:07:27 DEBUG : 01_DES_MATERIAL/TEST/.DS_Store: Excluded
2022/10/09 13:07:27 DEBUG : S3 bucket ba-despues-2022: Waiting for checks to finish
2022/10/09 13:07:27 DEBUG : S3 bucket ba-despues-2022: Waiting for transfers to finish
2022/10/09 13:07:28 DEBUG : 01_DES_MATERIAL/TEST/2022-09-11 03.30.07.mp4: md5 = 9424a0f7d030adbde782de24f1060bcd (Local file system at /Users/rodrigo/Dropbox/Trabajo/Brujazul/2022/Despues)
2022/10/09 13:07:28 DEBUG : 01_DES_MATERIAL/TEST/2022-09-11 03.30.07.mp4: md5 = a11f8672924d577b8bc29c847aab0b5a (S3 bucket ba-despues-2022)
2022/10/09 13:07:28 ERROR : 01_DES_MATERIAL/TEST/2022-09-11 03.30.07.mp4: corrupted on transfer: md5 hash differ "9424a0f7d030adbde782de24f1060bcd" vs "a11f8672924d577b8bc29c847aab0b5a"

My guess is that the IDrive implementation in rclone doesn't work when using SSE-C encryption, so that is my question, is it compatible, if so, how should I use it to be able to pass my own keys?

asdffdsa · October 9, 2022, 7:30pm

hi,

sse-c + idrive is working for me.

i suggest

create a new empty bucket
write a simple script to copy a single file into that empty bucket.

here is the config

[idrive_with_ssec]
type = s3
provider = IDrive
access_key_id = redacted
secret_access_key = redacted
endpoint = o0e2.va12.idrivee2-9.com
sse_customer_algorithm = AES256
sse_customer_key = redacted

[idrive_without_ssec]
type = s3
provider = IDrive
access_key_id = redacted
secret_access_key = redacted
endpoint = o0e2.va12.idrivee2-9.com

here is the script + lightly edited output

# upload local file to idrive with sse-c
+ rclone copy ./source/file.ext idrive_with_ssec:bucket -vv
DEBUG : rclone: Version "v1.59.2" starting with parameters ["rclone" "copy" "./source/file.ext" "idrive_with_ssec:bucket" "-vv"]
DEBUG : file.ext: Need to transfer - File not found at Destination
DEBUG : file.ext: md5 = c7f5af9b93f5aa17934c84ad53fd2cea OK
INFO  : file.ext: Copied (new)

# download from idrive with sse-c
+ rclone copy idrive_with_ssec:bucket ./dest -vv
DEBUG : rclone: Version "v1.59.2" starting with parameters ["rclone" "copy" "idrive_with_ssec:bucket" "./dest" "-vv"]
DEBUG : file.ext: md5 = c7f5af9b93f5aa17934c84ad53fd2cea OK
INFO  : file.ext: Copied (new)

# fail to download from idrive without sse-c
+ rclone copy idrive_without_ssec:bucket ./dest -vv
DEBUG : rclone: Version "v1.59.2" starting with parameters ["rclone" "copy" "idrive_without_ssec:bucket" "./dest" "-vv" "--retries=1"]
ERROR : file.ext: Failed to copy: failed to open source object: InvalidRequest: The object was stored using a form of Server Side Encryption. The correct parameters must be provided to retrieve the object.
	status code: 400, request id: 171C81029EA0F835, host id:

rgomezc · October 9, 2022, 9:00pm

Thanks for the reply.

I haven't been able to make it work. I have created new buckets, new access keys, but I always get the same symptoms as originally posted. Not really sure what I am doing differently for this to fail... I'll make some more tests tomorrow.

Question: did you enable the "default encryption" toggle on the iDrive interface when creating the new bucket? I have tried with both, and changed that setting afterwards also, with no difference, but just thought to ask.

Thanks for your help!

asdffdsa · October 9, 2022, 9:07pm

no idea what that is.
i have a free idrive account that i use for testing only.

it has been a long time since i logged into the web interface.
but since you asked, just now i logged in and enabled default encryption for my idrive bucket.
i re-ran the same exact script and got the same exact output.
so that does not seem to be the issue.

asdffdsa · October 9, 2022, 9:29pm

might try using --dump=headers

idrive with sse-c

DEBUG : GET /bucket/file.ext HTTP/1.1
Host: o0e2.va12.idrivee2-9.com
User-Agent: rclone/v1.59.2
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20221009T212553Z
X-Amz-Server-Side-Encryption-Customer-Algorithm: AES256
X-Amz-Server-Side-Encryption-Customer-Key: redacted
X-Amz-Server-Side-Encryption-Customer-Key-Md5: redacted

DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
DEBUG : HTTP RESPONSE (req 0xc00023d600)
DEBUG : HTTP/1.1 200 OK

idrive without sse-c

DEBUG : GET /bucket/file.ext HTTP/1.1
Host: o0e2.va12.idrivee2-9.com
User-Agent: rclone/v1.59.2
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20221009T212553Z

DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
DEBUG : HTTP RESPONSE (req 0xc0007e8300)
DEBUG : HTTP/1.1 400 Bad Request

rgomezc · October 10, 2022, 3:35pm

Hello,

I added the --dump=headers flag and there is something I don't know if it's correct:

2022/10/10 10:26:51 DEBUG : HTTP REQUEST (req 0x140001ae500)
2022/10/10 10:26:51 DEBUG : PUT /despues-2022/01_DES_MATERIAL/TEST/2022-09-11%2003.30.07.mp4 HTTP/1.1
Host: v4t2.da.idrivee2-35.com
User-Agent: rclone/v1.59.2
Content-Length: 1795712
Authorization: XXXX
Content-Md5: lCSg99Awrb3ngt4k8QYLzQ==
Content-Type: video/mp4
X-Amz-Acl: private
X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
X-Amz-Date: 20221010T152651Z
X-Amz-Meta-Md5chksum: lCSg99Awrb3ngt4k8QYLzQ==
X-Amz-Meta-Mtime: 1662885007
X-Amz-Server-Side-Encryption-Customer-Algorithm: AES256
X-Amz-Server-Side-Encryption-Customer-Key: <redacted: should it be a base64 encoding of the key or something derived from it?>
X-Amz-Server-Side-Encryption-Customer-Key-Md5: <redacted: base64 of md5>
Accept-Encoding: gzip

2022/10/10 10:26:51 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/10/10 10:26:51 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/10/10 10:26:51 DEBUG : HTTP RESPONSE (req 0x140001ae500)
2022/10/10 10:26:51 DEBUG : HTTP/1.1 403 Forbidden
Content-Length: 375
Accept-Ranges: bytes
Connection: keep-alive
Content-Security-Policy: block-all-mixed-content
Content-Type: application/xml
Date: Mon, 10 Oct 2022 15:26:51 GMT
Server: nginx
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
Vary: Accept-Encoding
X-Amz-Request-Id: 171CBEACDB2F1AAB
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block

What I don't know is: does the Key should be exactly the text I used, or something else made using as base the text I set? that's the only thing I see as not expected, but being the first time I use SSE-C wherever, I don't know what to expect.

Edit: another difference is the UNSIGNED-PAYLOAD in the header. In the headers sent by @asdffdsa there is the hash of the content, here it's not. This are big files (and will be bigger on the real-life scenario)... don't know if that could have something to do? I will test with smaller files and see what happens...

Edit 2: I tried uploading a small text file and I still get the same UNSIGNED-PAYLOAD header and the same error. Removing the key and the algorithm from rclone config works: the file is uploaded, but that header still reads the same: UNSIGNED-PAYLOAD:

2022/10/10 10:41:32 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/10/10 10:41:32 DEBUG : HTTP REQUEST (req 0x14000352300)
2022/10/10 10:41:32 DEBUG : PUT /despues-2022/01_DES_MATERIAL/TEST/exclusiones.txt HTTP/1.1
Host: v4t2.da.idrivee2-35.com
User-Agent: rclone/v1.59.2
Content-Length: 26
Authorization: XXXX
Content-Md5: PuMtW/dvoMbrZBn3CTlLFQ==
Content-Type: text/plain; charset=utf-8
X-Amz-Acl: private
X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
X-Amz-Date: 20221010T154132Z
X-Amz-Meta-Mtime: 1665256222.655937002
Accept-Encoding: gzip

I don't know if this could be the, or at least part of, the problem.

asdffdsa · October 10, 2022, 3:58pm

can test with this key 11111111111111111111111111111111

--- always make sure the dest bucket/dir is empty.
--- copy a single small text file and post the full debug log.

rgomezc · October 10, 2022, 5:20pm

Hello,

With that key it works!

2022/10/10 11:31:04 DEBUG : --max-age 99y to 1923-11-04 09:31:04.952786 -0700 MST m=-3122063999.988153333
2022/10/10 11:31:04 DEBUG : rclone: Version "v1.59.2" starting with parameters ["./rclone-arm64" "copy" "../" "e2-ba-despues:despues-2022" "--config" "./rclone.config" "-P" "--fast-list" "--max-age" "99y" "--exclude-from" "./exclusiones.txt" "--checkers=24" "--stats=1s" "--log-file=/Users/rodrigo/Dropbox/Trabajo/Brujazul/2022/Despues/99_DES_UTILERIAS_NUBE/logs/backup_completo-2022-10-10T113104.log" "-vv" "--ask-password=false" "--dump=headers"]
2022/10/10 11:31:04 DEBUG : Creating backend with remote "../"
2022/10/10 11:31:04 DEBUG : Using config file from "/Users/rodrigo/Dropbox/Trabajo/Brujazul/2022/Despues/99_DES_UTILERIAS_NUBE/rclone.config"
2022/10/10 11:31:04 DEBUG : fs cache: renaming cache item "../" to be canonical "/Users/rodrigo/Dropbox/Trabajo/Brujazul/2022/Despues"
2022/10/10 11:31:04 DEBUG : Creating backend with remote "e2-ba-despues:despues-2022"
2022/10/10 11:31:04 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2022/10/10 11:31:04 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2022/10/10 11:31:04 DEBUG : .DS_Store: Excluded
2022/10/10 11:31:04 DEBUG : 99_DES_UTILERIAS_NUBE: Excluded
2022/10/10 11:31:04 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/10/10 11:31:04 DEBUG : HTTP REQUEST (req 0x1400062e200)
2022/10/10 11:31:04 DEBUG : GET /despues-2022?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix= HTTP/1.1
Host: v4t2.da.idrivee2-35.com
User-Agent: rclone/v1.59.2
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20221010T163104Z
Accept-Encoding: gzip

2022/10/10 11:31:04 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/10/10 11:31:05 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/10/10 11:31:05 DEBUG : HTTP RESPONSE (req 0x1400062e200)
2022/10/10 11:31:05 DEBUG : HTTP/1.1 200 OK
Content-Length: 298
Accept-Ranges: bytes
Connection: keep-alive
Content-Security-Policy: block-all-mixed-content
Content-Type: application/xml
Date: Mon, 10 Oct 2022 16:31:05 GMT
Server: nginx
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
Vary: Accept-Encoding
X-Amz-Request-Id: 171CC22E0277B2A9
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block

2022/10/10 11:31:05 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/10/10 11:31:05 DEBUG : 01_DES_MATERIAL/.DS_Store: Excluded
2022/10/10 11:31:05 DEBUG : 01_DES_MATERIAL/TEST/.DS_Store: Excluded
2022/10/10 11:31:05 DEBUG : S3 bucket despues-2022: Waiting for checks to finish
2022/10/10 11:31:05 DEBUG : S3 bucket despues-2022: Waiting for transfers to finish
2022/10/10 11:31:05 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/10/10 11:31:05 DEBUG : HTTP REQUEST (req 0x140001ae600)
2022/10/10 11:31:05 DEBUG : PUT /despues-2022/01_DES_MATERIAL/TEST/exclusiones.txt HTTP/1.1
Host: v4t2.da.idrivee2-35.com
User-Agent: rclone/v1.59.2
Content-Length: 26
Authorization: XXXX
Content-Md5: PuMtW/dvoMbrZBn3CTlLFQ==
Content-Type: text/plain; charset=utf-8
X-Amz-Acl: private
X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
X-Amz-Date: 20221010T163105Z
X-Amz-Meta-Md5chksum: PuMtW/dvoMbrZBn3CTlLFQ==
X-Amz-Meta-Mtime: 1665256222.655937002
X-Amz-Server-Side-Encryption-Customer-Algorithm: AES256
X-Amz-Server-Side-Encryption-Customer-Key: MTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTE=
X-Amz-Server-Side-Encryption-Customer-Key-Md5: GoDL8oWeAZVZNl1r5Hh5Tg==
Accept-Encoding: gzip

2022/10/10 11:31:05 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/10/10 11:31:05 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/10/10 11:31:05 DEBUG : HTTP RESPONSE (req 0x140001ae600)
2022/10/10 11:31:05 DEBUG : HTTP/1.1 200 OK
Content-Length: 0
Accept-Ranges: bytes
Connection: keep-alive
Content-Security-Policy: block-all-mixed-content
Date: Mon, 10 Oct 2022 16:31:05 GMT
Etag: "9857337eea1ac72579c34750fa4c44b4"
Server: nginx
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
Vary: Accept-Encoding
X-Amz-Request-Id: 171CC22E05CB02D5
X-Amz-Server-Side-Encryption-Customer-Algorithm: AES256
X-Amz-Server-Side-Encryption-Customer-Key-Md5: GoDL8oWeAZVZNl1r5Hh5Tg==
X-Amz-Version-Id: null
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block

2022/10/10 11:31:05 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/10/10 11:31:05 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/10/10 11:31:05 DEBUG : HTTP REQUEST (req 0x140009f8400)
2022/10/10 11:31:05 DEBUG : HEAD /despues-2022/01_DES_MATERIAL/TEST/exclusiones.txt HTTP/1.1
Host: v4t2.da.idrivee2-35.com
User-Agent: rclone/v1.59.2
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20221010T163105Z
X-Amz-Server-Side-Encryption-Customer-Algorithm: AES256
X-Amz-Server-Side-Encryption-Customer-Key: MTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTE=
X-Amz-Server-Side-Encryption-Customer-Key-Md5: GoDL8oWeAZVZNl1r5Hh5Tg==

2022/10/10 11:31:05 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/10/10 11:31:05 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/10/10 11:31:05 DEBUG : HTTP RESPONSE (req 0x140009f8400)
2022/10/10 11:31:05 DEBUG : HTTP/1.1 200 OK
Content-Length: 26
Accept-Ranges: bytes
Connection: keep-alive
Content-Security-Policy: block-all-mixed-content
Content-Type: text/plain; charset=utf-8
Date: Mon, 10 Oct 2022 16:31:05 GMT
Etag: "9857337eea1ac72579c34750fa4c44b4"
Last-Modified: Mon, 10 Oct 2022 16:31:05 GMT
Server: nginx
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
Vary: Accept-Encoding
X-Amz-Meta-Md5chksum: PuMtW/dvoMbrZBn3CTlLFQ==
X-Amz-Meta-Mtime: 1665256222.655937002
X-Amz-Request-Id: 171CC22E08459144
X-Amz-Server-Side-Encryption-Customer-Algorithm: AES256
X-Amz-Server-Side-Encryption-Customer-Key-Md5: GoDL8oWeAZVZNl1r5Hh5Tg==
X-Amz-Version-Id: null
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block

2022/10/10 11:31:05 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/10/10 11:31:05 DEBUG : 01_DES_MATERIAL/TEST/exclusiones.txt: md5 = 3ee32d5bf76fa0c6eb6419f709394b15 OK
2022/10/10 11:31:05 INFO  : 01_DES_MATERIAL/TEST/exclusiones.txt: Copied (new)
2022/10/10 11:31:05 INFO  : 
Transferred:   	         26 B / 26 B, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         0.2s

I will try with that password the other files, or a password of the same length...

Thanks!

asdffdsa · October 10, 2022, 6:18pm

i always use a random string of 32 ASCII characters
such as xkC8vPkF?Yn2khkg&qA!Rq-?SUYC3qUD

can get more info at
https://github.com/rclone/rclone/issues/6400

rgomezc · October 10, 2022, 6:29pm

The previous passwords I had used were probably less than 32 characters. So I will try with that and let you know.

ncw · October 11, 2022, 10:41am

The sse key needs to be exactly 32 characters.

Note that we merged a way if supplying it in base64 recently which may how you have the key?

github.com/rclone/rclone

You cannot use a full binary value for sse_customer_key

opened 07:16PM - 27 Aug 22 UTC

closed 04:28PM - 17 Sep 22 UTC

taxilian

question

#### The associated forum post URL from `https://forum.rclone.org` https://forum.rclone.org/t/cant-use-sse-customer-key-with-non-printable-charactersin-config-file/32667 #### What is your current rclone version (output from `rclone version`)? tried with v1.56.2 and v1.50.2 #### What problem are you are trying to solve? There does not seem to be any way to put a non-printable sse_customer_key in the config file -- also, that key should probably be obscured anyway. #### How do you think rclone should be changed to solve that? The easiest method that occurs to me is to just allow it to be obscured; since you can obscure a non-printable key and it results in printable characters that should do it. The other option I see would be to detect in some way if it is a base64 encoded value, e.g. "this is longer than 32 bytes and it's an AES256 key, so it must be b64 encoded" or supporting `base64:<b64 encoded customer key>` Seems like it should be a simple enough change. #### How to use GitHub * Please use the 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to show that you are affected by the same issue. * Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue. * Subscribe to receive notifications on status change and new comments.

The fix for that is in the latest beta

rgomezc · October 11, 2022, 5:52pm

Hello,

The problem was in fact the key: previous ones were not 32 characters, don't remember if they were less or more, but using a 32 length string made the problem go away.

I don't need the base64 encoding for the key right now, but it's good to know that it is possible.

Thanks for your help!

system · November 10, 2022, 5:53pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Is SSE-C supported when using iDrive e2

What is the problem you are having with rclone?

Run the command 'rclone version' and share the full output of the command.

Which cloud storage system are you using? (eg Google Drive)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

The rclone config contents with secrets removed.

A log from the command with the -vv flag

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

A log from the command with the `-vv` flag