rclone can't connect to my Remarkable 2 device via ssh, though ssh directly works fine. The error message is just error couldn't connect SSH: ssh: handshake failed: EOF.
I've tried:
rclone version from apt and 1.61.1 .deb download from rclone.org
auth with password
auth with RSA 2048 key, passphrase-encrypted and not
auth with ED25519 key, passphrase-encrypted and not
ssh-agent option in rclone enabled and not
interactive mode
replacing /bin/ssh with a script that calls ssh -vvv and logs output to a file
removing the $SSH_AUTH_SOCK env var so I'm sure it's not using the agent.
I'm not sure how to go further, since there is no rclone option to get ssh connection debug logs.
Run the command 'rclone version' and share the full output of the command.
Which cloud storage system are you using? (eg Google Drive)
SFTP to a Remarkable 2 device. Direct SSH access works fine, with any of my keys or via password. Verbose SSH output tells me Remote protocol version 2.0, remote software version dropbear_2020.81
The command you were trying to run (eg rclone copy /tmp remote:tmp)
I'm posting as Suspected Bug because this is the simplest operation and config I can imagine, on the most common distro. If it's user error, it's worth improving the error message.
I totally agree... I just can't imagine what! I tried with various --dump flags, but got the same output. Even --dump openfiles to see if I could guess which one was getting a premature EOF.
Here's the log output from sshd on the server (good idea!), for one connection attempt:
Dec 27 11:00:02 reMarkable systemd[1]: Condition check resulted in SSH Key Generation being skipped.
-- Subject: A start job for unit dropbearkey.service has finished successfully
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- A start job for unit dropbearkey.service has finished successfully.
--
-- The job identifier is 3372.
Dec 27 11:00:03 reMarkable systemd[1]: Started SSH Per-Connection Server (10.10.10.210:56204).
-- Subject: A start job for unit dropbear@40-10.10.10.121:22-10.10.10.210:56204.service has finished successfully
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- A start job for unit dropbear@40-10.10.10.121:22-10.10.10.210:56204.service has finished successfully.
--
-- The job identifier is 3297.
Dec 27 11:00:03 reMarkable dropbear[4652]: Child connection from ::ffff:10.10.10.210:56204
Dec 27 11:00:03 reMarkable dropbear[4652]: Exit before auth from <::ffff:10.10.10.210:56204>: Failed assertion (../dropbear-2020.81/rsa.c:164): `key != NULL'
Dec 27 11:00:03 reMarkable systemd[1]: dropbear@40-10.10.10.121:22-10.10.10.210:56204.service: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- The unit dropbear@40-10.10.10.121:22-10.10.10.210:56204.service has successfully entered the 'dead' state.
Here is journalctl output from a successful SSH connection attempt (from the same client machine):
Dec 27 11:01:58 reMarkable systemd[1]: dropbear@30-2a02:8109:a280:2288:72f7:54ff:fe76:b4b8:22-2a02:8109:a280:2288:42bf:a71f:2bb3:60ae:35620.service: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- The unit dropbear@30-2a02:8109:a280:2288:72f7:54ff:fe76:b4b8:22-2a02:8109:a280:2288:42bf:a71f:2bb3:60ae:35620.service has successfully entered the 'dead' state.
Dec 27 11:02:03 reMarkable systemd[1]: Condition check resulted in SSH Key Generation being skipped.
-- Subject: A start job for unit dropbearkey.service has finished successfully
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- A start job for unit dropbearkey.service has finished successfully.
--
-- The job identifier is 3448.
Dec 27 11:02:03 reMarkable systemd[1]: Started SSH Per-Connection Server ([2a02:8109:a280:2288:42bf:a71f:2bb3:60ae]:35490).
-- Subject: A start job for unit dropbear@41-2a02:8109:a280:2288:72f7:54ff:fe76:b4b8:22-2a02:8109:a280:2288:42bf:a71f:2bb3:60ae:35490.service has finished successfully
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- A start job for unit dropbear@41-2a02:8109:a280:2288:72f7:54ff:fe76:b4b8:22-2a02:8109:a280:2288:42bf:a71f:2bb3:60ae:35490.service has finished successfully.
--
-- The job identifier is 3373.
Dec 27 11:02:03 reMarkable dropbear[4710]: Child connection from 2a02:8109:a280:2288:42bf:a71f:2bb3:60ae:35490
Dec 27 11:02:03 reMarkable dropbear[4710]: Pubkey auth succeeded for 'root' with key sha1!! 09:a0:c7:1f:ca:a2:a0:bb:7f:96:a6:49:0e:a6:67:db:2d:4c:ac:6e from 2a02:8109:a280:2288:42bf:a71f:2bb3:60ae:35490
So there is something different between the ssh-client connection and the rclone connection that causes dropbear to run the assertion check at dropbear-2020.81/rsa.c:164 ( dropbear/rsa.c at d852d69b50187dd81d424846e7dc677ec57e2d4f · mkj/dropbear · GitHub ). Without knowing the SSH connection options rclone uses it's tricky to figure out what the difference is.
The server is running with /usr/sbin/dropbear -i -r /etc/dropbear/dropbear_ed25519_host_key -B ... which is strange, because -r should indicate the server's RSA host key.
Hypothesis: openssh expects an ed25519 host key, so that -r problem is a non-issue. But rclone expects an rsa host key, so it breaks.
in the version of dropbear in use, -r is agnostic to the type of host key used. The default is an array of DSA, RSA, etc. So there's no special header mismatch where eg the key is presented as RSA but is really ed25519. It's presented as Ed25519 and that's what it is.
if I change how dropbear is launched to include an RSA host key, rclone works without complaint.
I've filed a support issue with Remarkable, since it's their implementation of dropbear at the root of this.
Now the question for rclone:
Is it a bug that rclone requires an RSA key and breaks with an unclear error message when one isn't available? IMO yes, the requirement should at least be documented and a clearer error message presented. But I'm not a maintainer.
Rclone is quite capable of using a ed25519 key, I think it is just querying the RSA key first. I presume dropbear should be returning key not found or something like that to allow rclone to move on to the next one?
