Seafile request: add Client Certificate Authentication

bomzh · June 26, 2024, 2:58pm

Hello!

I know there's support for Client Certificate Authentication (PEM) for HTTP method. It would be nice to have similar functionality for Seafile method since it also uses HTTP to communication between rclone <-> seafile.

We're using Seafile server behind Apache proxy that uses client https certificates and right now we're forced to do turn off client certificate validation on Apache proxy for some clients that use Rclone.

Thanks for great software!

asdffdsa · June 26, 2024, 3:23pm

welcome to the forum,

--client-cert string                 Client SSL certificate (PEM) for mutual TLS auth
--client-key string                  Client SSL private key (PEM) for mutual TLS auth

those flags did not work?
can you post a rclone debug log?

bomzh · June 26, 2024, 3:50pm

No, these flags have no effect when you attempt to access Seafile behind proxy that requires client SSL certificate (and key and optional CA). It took me several hours trying various options until I recognized that rclone does not send client certificate.

asdffdsa · June 26, 2024, 3:51pm

ok, you did not mention that or the flags you tried, what version of rclone, commands, debug log, and what those options were.
makes it hard to help and hard to understand, without having to pull it from you...

--no-check-certificate=true
that does not work?

what other tools have you tested, for example, curl?
and what was the exact command that worked?

ncw · June 26, 2024, 5:24pm

asdffdsa:

--client-cert string                 Client SSL certificate (PEM) for mutual TLS auth
--client-key string                  Client SSL private key (PEM) for mutual TLS auth

those flags did not work?

Those flags should be working...

Looking at the seafile code it does the right things in order to support it.

Can you try a simpler method on the proxy, something like

rclone copyurl --stdout https://rclone.com --client-cert string --client-key string

And see if you can get that to work?

bomzh · June 26, 2024, 7:19pm

Thanks all for responding.
Well, first of all let me point here (maybe this is outdated information):

github.com/rclone/rclone

Rclone ignores `--ca-cert <path_to_cert>` option (`ftp` remote or `serve sftp`)

opened 03:23PM - 06 Jan 23 UTC

StanleyP

enhancement Remote: SFTP

#### What is the problem you are having with rclone? Rclone seems to ignore m…y `--ca-cert <path_to_cert>` option while using `ftp` remote either directly or via `serve sftp`. But as soon as I add given CA certificate to `/usr/local/share/ca-certificates` and run `update-ca-certificates` everything works OK. My main usecase is that I want to expose FTPS server as SFTP server via Rclone. I'm using `--auth-proxy`. Ideal solution for me would be if I could specify `--ca-cert` dynamically from auth proxy. Would that be possible? #### What is your rclone version (output from `rclone version`) `1.61.1` #### Which OS you are using and how many bits (e.g. Windows 7, 64 bit) `Ubuntu 20.04, 64bit` #### Which cloud storage system are you using? (e.g. Google Drive) `ftps` #### The command you were trying to run (e.g. `rclone copy /tmp remote:tmp`) ``` rclone -vv --ca-cert /conf/ssl-cert-san.pem ls ftps: rclone -vv ls ftps: --ca-cert /conf/ssl-cert-san.pem rclone -vv --ca-cert /conf/ssl-cert-san.pem serve sftp ftps: ``` config ``` [ftps] type = ftp host = XX.XX.XXX.XX user = ftpuser port = 3021 pass = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX tls = true disable_tls13 = true ``` #### A log from the command with the `-vv` flag (e.g. output from `rclone -vv copy /tmp remote:tmp`) ``` rclone -vv --ca-cert /conf/ssl-cert-san.pem ls ftps: 2023/01/06 14:59:02 DEBUG : rclone: Version "v1.60.1" starting with parameters ["rclone" "-vv" "--ca-cert" "/conf/ssl-cert-san.pem" "ls" "ftps:"] 2023/01/06 14:59:02 DEBUG : Creating backend with remote "ftps:" 2023/01/06 14:59:02 DEBUG : Using config file from "/root/.config/rclone/rclone.conf" 2023/01/06 14:59:02 DEBUG : ftps://XX.XX.XXX.XX:3021: Connecting to FTP server 2023/01/06 14:59:02 DEBUG : ftps://XX.XX.XXX.XX:3021: dial("tcp","XX.XX.XXX.XX:3021") 2023/01/06 14:59:02 DEBUG : ftps://XX.XX.XXX.XX:3021: > dial: conn=*tls.Conn, err=<nil> 2023/01/06 14:59:02 Failed to create file system for "ftps:": NewFs: failed to make FTP connection to "XX.XX.XXX.XX:3021": x509: certificate signed by unknown authority ``` #### How to use GitHub * Please use the 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to show that you are affected by the same issue. * Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue. * Subscribe to receive notifications on status change and new comments.

@ncw says there the following:

--ca-cert only works with the HTTP backends at the moment.

Ok, lets assume that this is not true anymore and that --ca-cert and --client-key and --client-cert are working in the last version.

So here's my findings using CURL and RCLONE on the same URL where Seafile is proxied from:

1) CURL w/o client certificate:

1) CURL w/o client certificate:
$ curl -v --head  https://my.cloud.com/
*   Trying 1.2.3.4:443...
* Connected to my.cloud.com (1.2.3.4) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=my.cloud.com
*  start date: Jun 13 18:35:38 2024 GMT
*  expire date: Sep 11 18:35:37 2024 GMT
*  subjectAltName: host "my.cloud.com" matched cert's "my.cloud.com"
*  issuer: C=US; O=Let's Encrypt; CN=R11
*  SSL certificate verify ok.
> HEAD / HTTP/1.1
> Host: my.cloud.com
> User-Agent: curl/7.74.0
> Accept: */*
> 
* TLSv1.2 (IN), TLS handshake, Hello request (0):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS alert, handshake failure (552):
* OpenSSL SSL_read: error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure, errno 0
* Closing connection 0
curl: (56) OpenSSL SSL_read: error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure, errno 0

We can clearly see that CURL fails with routines:ssl3_read_bytes:sslv3 alert handshake failure, errno 0, because no client certificate supplied to CURL.

2) CURL w/ client certificate same URL:

$ curl -v --head --cacert /home/user/.config/ssl/CA.crt --cert /home/user/.config/ssl/client.crt --key /home/user/.config/ssl/client.key https://my.cloud.com/
*   Trying 1.2.3.4:443...
* Connected to my.cloud.com (1.2.3.4) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: CA.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=my.cloud.com
*  start date: Jun 13 18:35:38 2024 GMT
*  expire date: Sep 11 18:35:37 2024 GMT
*  subjectAltName: host "my.cloud.com" matched cert's "my.cloud.com"
*  issuer: C=US; O=Let's Encrypt; CN=R11
*  SSL certificate verify ok.
> HEAD / HTTP/1.1
> Host: my.cloud.com
> User-Agent: curl/7.74.0
> Accept: */*
> 
* TLSv1.2 (IN), TLS handshake, Hello request (0):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 302 Found
HTTP/1.1 302 Found
< Date: Wed, 26 Jun 2024 18:55:25 GMT
Date: Wed, 26 Jun 2024 18:55:25 GMT
< Server: nginx
Server: nginx
< Content-Type: text/html; charset=utf-8
Content-Type: text/html; charset=utf-8
< Location: /accounts/login/?next=/
Location: /accounts/login/?next=/
< Vary: Accept-Language,Cookie
Vary: Accept-Language,Cookie
< Content-Language: en
Content-Language: en

< 
* Connection #0 to host my.cloud.com left intact

This works as expected - we can see server headers and all's fine

3) RCLONE with client cert:

$ rclone sync -vv --no-check-certificate --ca-cert $HOME/.config/ssl/CA.crt --client-cert $HOME/.config/ssl/client.crt --client-key $HOME/.config/ssl/client.key -P -vv $HOME/files/new/ super_cloud:SUPER-LIB/report/
2024/06/26 18:56:32 DEBUG : rclone: Version "v1.53.3-DEV" starting with parameters ["rclone" "sync" "-vv" "--no-check-certificate" "--ca-cert" "/home/user/.config/ssl/CA.crt" "--client-cert" "/home/user/.config/ssl/client.crt" "--client-key" "/home/user/.config/ssl/client.key" "-P" "-vv" "/home/user/files/new/" "super_cloud:SUPER-LIB/report/"]
2024/06/26 18:56:32 DEBUG : Creating backend with remote "/home/user/files/new/"
2024/06/26 18:56:32 DEBUG : Using config file from "/home/user/.config/rclone/rclone.conf"
2024/06/26 18:56:32 DEBUG : Creating backend with remote "super_cloud:SUPER-LIB/report/"
2024/06/26 18:56:32 Failed to create file system for "super_cloud:SUPER-LIB/report/": failed to get server info: Get "https://my.cloud.com/api2/server-info/": local error: tls: no renegotiation

and Apache's log (where Seafile proxy lives) shows:

[Wed Jun 26 20:56:32.215077 2024] [ssl:error] [pid 70800:tid 140050134124288] [client 1.1.1.1:58668] AH02261: Re-negotiation handshake failed
[Wed Jun 26 20:56:32.215142 2024] [ssl:error] [pid 70800:tid 140050134124288] SSL Library Error: error:14094153:SSL routines:ssl3_read_bytes:no renegotiation

So I suppose this means that RCLONE did not send any client certificate, SSL/TLS connection failed.

4) RCLONE w/o client cert (Seafile Apache proxy has "SSLVerifyClient none" set):

$ rclone sync -vv --no-check-certificate --ca-cert $HOME/.config/ssl/CA.crt --client-cert $HOME/.config/ssl/client.crt --client-key $HOME/.config/ssl/client.key -P -vv $HOME/files/new/ super_cloud:SUPER-LIB/report/
2024/06/26 18:58:59 DEBUG : rclone: Version "v1.53.3-DEV" starting with parameters ["rclone" "sync" "-vv" "--no-check-certificate" "--ca-cert" "/home/user/.config/ssl/CA.crt" "--client-cert" "/home/user/.config/ssl/client.crt" "--client-key" "/home/user/.config/ssl/client.key" "-P" "-vv" "/home/user/files/new/" "super_cloud:SUPER-LIB/report/"]
2024/06/26 18:58:59 DEBUG : Creating backend with remote "/home/user/files/new/"
2024/06/26 18:58:59 DEBUG : Using config file from "/home/user/.config/rclone/rclone.conf"
2024/06/26 18:58:59 DEBUG : Creating backend with remote "super_cloud:SUPER-LIB/report/"
2024/06/26 18:58:59 DEBUG : Seafile server version 11.0.8
2024/06/26 18:59:00 DEBUG : fs cache: renaming cache item "super_cloud:SUPER-LIB/report/" to be canonical "super_cloud:SUPER-LIB/report"
2024-06-26 18:59:00 DEBUG : file_20240620.csv: Sizes identical
2024-06-26 18:59:00 DEBUG : file_20240620.csv: Unchanged skipping
2024-06-26 18:59:00 DEBUG : file_20240619.csv: Sizes identical
2024-06-26 18:59:00 DEBUG : file_20240619.csv: Unchanged skipping
2024-06-26 18:59:00 DEBUG : seafile library 'SUPER-LIB' path 'report': Waiting for checks to finish
2024-06-26 18:59:00 DEBUG : seafile library 'SUPER-LIB' path 'report': Waiting for transfers to finish
2024-06-26 18:59:00 DEBUG : Waiting for deletions to finish
2024-06-26 18:59:00 INFO  : There was nothing to transfer
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks:                 2 / 2, 100%
Elapsed time:         0.3s
2024/06/26 18:59:00 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks:                 2 / 2, 100%
Elapsed time:         0.3s

2024/06/26 18:59:00 DEBUG : 5 go routines active

Works! Because server does not require client certificate (SSLVerifyClient none).

And I also tried latest version of RCLONE (v1.67.0) - it also fails same way as in step 4.

Can you try a simpler method on the proxy, something like
This also fails:

$ rclone copyurl --stdout -vv https://my.cloud.com/ --client-cert $HOME/.config/ssl/client.crt --client-key $HOME/.config/ssl/client.key
2024/06/26 19:17:32 DEBUG : rclone: Version "v1.53.3-DEV" starting with parameters ["rclone" "copyurl" "--stdout" "-vv" "https://my.cloud.com/" "--client-cert" "/home/user/.config/ssl/client.crt" "--client-key" "/home/user/.config/ssl/client.key"]
2024/06/26 19:17:32 ERROR : Attempt 1/3 failed with 1 errors and: Get "https://my.cloud.com/": local error: tls: no renegotiation
2024/06/26 19:17:32 ERROR : Attempt 2/3 failed with 1 errors and: Get "https://my.cloud.com/": local error: tls: no renegotiation
2024/06/26 19:17:32 ERROR : Attempt 3/3 failed with 1 errors and: Get "https://my.cloud.com/": local error: tls: no renegotiation
2024/06/26 19:17:32 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.0s

2024/06/26 19:17:32 DEBUG : 2 go routines active
2024/06/26 19:17:32 Failed to copyurl: Get "https://my.cloud.com/": local error: tls: no renegotiation

asdffdsa · June 26, 2024, 7:39pm

why are you testing a development version, almost four years old??
please, can you test latest stable and post the full output.
and post the rest of the help and support template.

rclone version
rclone config redacted

i am not expert, maybe it does not matter
but can you test rclone copyurl with --ca-cert, same as you did with curl.

bomzh · June 26, 2024, 8:13pm

The version I'm using is the one provided with Debian-11 repository. I agree that it may be outdated, but latest version (v1.67.0) shows me same behaviour.

but can you test rclone copyurl with --ca-cert, same as you did with curl.

I did tests with latest rclone v1.67.0 and here's what I get:

With --ca-cert --client-cert --client-key:
ERROR : Attempt 3/3 failed with 1 errors and: Get "https://my.cloud.com/": tls: failed to verify certificate: x509: certificate signed by unknown authority
With --ca-cert --client-cert --client-key AND --no-check-certificate=true:

ERROR : Attempt 3/3 failed with 1 errors and: Get "https://my.cloud.com/": local error: tls: no renegotiation

Both above results using latest version.

And my rclone config is pretty straight-forward:

[super_cloud]
type = seafile
url = https://my.cloud.com/
user = my-user
pass = 81txwPxxxkAusiyh34h
2fa = false

asdffdsa · June 26, 2024, 8:18pm

all i am asking for is complete debug logs, not one-line snippets.

bomzh · June 26, 2024, 8:24pm

Sure, if that helps:

$ /tmp/rclone/rclone-v1.67.0-linux-amd64/rclone copyurl --stdout -vv https://my.cloud.com/ --client-cert $HOME/.config/ssl/client.crt --client-key $HOME/.config/ssl/client.key --ca-cert $HOME/.config/ssl/CA.crt

2024/06/26 20:21:11 DEBUG : rclone: Version "v1.67.0" starting with parameters ["/tmp/rclone/rclone-v1.67.0-linux-amd64/rclone" "copyurl" "--stdout" "-vv" "https://my.cloud.com/" "--client-cert" "/home/user/.config/ssl/client.crt" "--client-key" "/home/user/.config/ssl/client.key" "--ca-cert" "/home/user/.config/ssl/CA.crt"]
2024/06/26 20:21:11 ERROR : Attempt 1/3 failed with 1 errors and: Get "https://my.cloud.com/": tls: failed to verify certificate: x509: certificate signed by unknown authority
2024/06/26 20:21:11 ERROR : Attempt 2/3 failed with 1 errors and: Get "https://my.cloud.com/": tls: failed to verify certificate: x509: certificate signed by unknown authority
2024/06/26 20:21:11 ERROR : Attempt 3/3 failed with 1 errors and: Get "https://my.cloud.com/": tls: failed to verify certificate: x509: certificate signed by unknown authority
2024/06/26 20:21:11 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.0s

2024/06/26 20:21:11 DEBUG : 4 go routines active
2024/06/26 20:21:11 Failed to copyurl: Get "https://my.cloud.com/": tls: failed to verify certificate: x509: certificate signed by unknown authority

$ /tmp/rclone/rclone-v1.67.0-linux-amd64/rclone copyurl --stdout -vv https://my.cloud.com/ --client-cert $HOME/.config/ssl/client.crt --client-key $HOME/.config/ssl/client.key --ca-cert $HOME/.config/ssl/CA.crt --no-check-certificate=true

2024/06/26 20:21:31 DEBUG : rclone: Version "v1.67.0" starting with parameters ["/tmp/rclone/rclone-v1.67.0-linux-amd64/rclone" "copyurl" "--stdout" "-vv" "https://my.cloud.com/" "--client-cert" "/home/user/.config/ssl/client.crt" "--client-key" "/home/user/.config/ssl/client.key" "--ca-cert" "/home/user/.config/ssl/CA.crt" "--no-check-certificate=true"]
2024/06/26 20:21:31 ERROR : Attempt 1/3 failed with 1 errors and: Get "https://my.cloud.com/": local error: tls: no renegotiation
2024/06/26 20:21:31 ERROR : Attempt 2/3 failed with 1 errors and: Get "https://my.cloud.com/": local error: tls: no renegotiation
2024/06/26 20:21:31 ERROR : Attempt 3/3 failed with 1 errors and: Get "https://my.cloud.com/": local error: tls: no renegotiation
2024/06/26 20:21:31 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.0s

2024/06/26 20:21:31 DEBUG : 4 go routines active
2024/06/26 20:21:31 Failed to copyurl: Get "https://my.cloud.com/": local error: tls: no renegotiation

system · August 25, 2024, 8:24pm

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.