Sftp with ssh tunnel

Help and Support
1

What is the problem you are having with rclone?

I am trying to copy files to a Synology using sftp using an ssh tunnel.
I can't get it to work.
What I understand is that it doesn't understand that the sftp-path-override should be applied to the target machine "Synology:Crono" and not to the server I use as an ssh tunnel

Run the command 'rclone version' and share the full output of the command.

rclone v1.66.0
- os/version: Microsoft Windows 11 Pro 23H2 (64 bit)
- os/kernel: 10.0.22631.3296 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.22.1
- go/linking: static
- go/tags: cmount

Which cloud storage system are you using?

Synology

The command you were trying to run

rclone.exe copy D:\prova.rtf Synology:Crono/POLICLINICO/RAW -vv --sftp-ssh "ssh xxxxx@login.roma1.infn.it" --sftp-path-override /volume2/Crono/POLICLINICO/RAW

Please run 'rclone config redacted' and share the full output.

[Synology]
type = sftp
host = XXX
user = XXX
pass = XXX
shell_type = unix
md5sum_command = md5sum
sha1sum_command = sha1sum

A log from the command that you were trying to run with the -vv flag

2024/03/28 10:15:46 DEBUG : rclone: Version "v1.66.0" starting with parameters ["C:\\Users\\mescs\\Rclone\\rclone.exe" "-vv" "--sftp-ssh" "ssh xxxx@login.roma1.infn.it" "--sftp-path-override" "/volume2/Crono/POLICLINICO/RAW" "copy" "D:\\prova.rtf" "Synology:Crono/POLICLINICO/RAW"]
2024/03/28 10:15:46 DEBUG : Creating backend with remote "D:\\prova.rtf"
2024/03/28 10:15:46 DEBUG : Using config file from "C:\\Users\\mescs\\AppData\\Roaming\\rclone\\rclone.conf"
2024/03/28 10:15:46 DEBUG : fs cache: adding new entry for parent of "D:\\prova.rtf", "//?/D:/"
2024/03/28 10:15:46 DEBUG : Creating backend with remote "Synology:Crono/POLICLINICO/RAW"
2024/03/28 10:15:46 DEBUG : Synology: detected overridden config - adding "{XjOCf}" suffix to name
2024/03/28 10:15:46 NOTICE: Synology{XjOCf}: --sftp-ssh is in use - ignoring user/host/port from config - set in the parameters to --sftp-ssh (remove them from the config to silence this warning)
2024/03/28 10:15:46 DEBUG : sftp://cobbs@xxx.xxx.x.xx:22/Crono/POLICLINICO/RAW: ssh external: creating additional session
2024/03/28 10:15:46 DEBUG : sftp://cobbs@xxx.xxx.x.xx:22/Crono/POLICLINICO/RAW: ssh external: running: ssh xxxx@login.roma1.infn.it -s sftp
2024/03/28 10:15:47 DEBUG : sftp://cobbs@xxx.xxx.x.xx:22/Crono/POLICLINICO/RAW: Shell type "unix" from config
2024/03/28 10:15:47 DEBUG : sftp://cobbs@xxx.xxx.x.xx:22/Crono/POLICLINICO/RAW: Failed to resolve path using RealPath: file does not exist
2024/03/28 10:15:47 DEBUG : sftp://cobbs@xxx.xxx.x.xx:22/Crono/POLICLINICO/RAW: Relative path joined with current directory to get absolute path "/afs/infn.it/roma1/user/xxxx/Crono/POLICLINICO/RAW"
2024/03/28 10:15:47 DEBUG : sftp://cobbs@xxx.xxx.x.xx:22/Crono/POLICLINICO/RAW: Using root directory "/afs/infn.it/roma1/user/xxxx/Crono/POLICLINICO/RAW"
2024/03/28 10:15:47 DEBUG : fs cache: renaming cache item "Synology:Crono/POLICLINICO/RAW" to be canonical "Synology{XjOCf}:Crono/POLICLINICO/RAW"
2024/03/28 10:15:47 DEBUG : prova.rtf: Need to transfer - File not found at Destination
2024/03/28 10:15:47 DEBUG : sftp://cobbs@xxx.xxx.x.xx:22/Crono/POLICLINICO/RAW: Connection OK after error: permission denied
2024/03/28 10:15:47 ERROR : prova.rtf: Failed to copy: Put mkParentDir failed: mkdir "/afs/infn.it/roma1/user/xxxx/Crono" failed: permission denied
2024/03/28 10:15:47 ERROR : Attempt 1 / 3 failed with 1 errors and: Put mkParentDir failed: mkdir "/afs/infn.it/roma1/user/xxxx/Crono" failed: permission denied
2024/03/28 10:15:47 DEBUG : prova.rtf: Need to transfer - File not found at Destination
2024/03/28 10:15:47 DEBUG : sftp://cobbs@xxx.xxx.x.xx:22/Crono/POLICLINICO/RAW: Connection OK after error: permission denied
2024/03/28 10:15:47 ERROR : prova.rtf: Failed to copy: Put mkParentDir failed: mkdir "/afs/infn.it/roma1/user/xxxx/Crono" failed: permission denied
2024/03/28 10:15:47 ERROR : Attempt 2 / 3 failed with 1 errors and: Put mkParentDir failed: mkdir "/afs/infn.it/roma1/user/xxxx/Crono" failed: permission denied
2024/03/28 10:15:47 DEBUG : prova.rtf: Need to transfer - File not found at Destination
2024/03/28 10:15:47 DEBUG : sftp://cobbs@xxx.xxx.x.xx:22/Crono/POLICLINICO/RAW: Connection OK after error: permission denied
2024/03/28 10:15:47 ERROR : prova.rtf: Failed to copy: Put mkParentDir failed: mkdir "/afs/infn.it/roma1/user/xxxx/Crono" failed: permission denied
2024/03/28 10:15:47 ERROR : Attempt 3 / 3 failed with 1 errors and: Put mkParentDir failed: mkdir "/afs/infn.it/roma1/user/xxxx/Crono" failed: permission denied
2024/03/28 10:15:47 INFO  :
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.6s

2024/03/28 10:15:47 DEBUG : 7 go routines active
2024/03/28 10:15:47 DEBUG : sftp://cobbs@xxx.xxx.x.xx:22/Crono/POLICLINICO/RAW: Closing 1 unused connections
2024/03/28 10:15:47 Failed to copy: Put mkParentDir failed: mkdir "/afs/infn.it/roma1/user/xxxx/Crono" failed: permission denied

It seems to me that it is mixed informations since:
"/afs/infn.it/roma1/user/xxxx/" is related to the server I use for ssh tunnel
while "/Crono/POLICLINICO/RAW" refers to the destination

here the two are mixed:

"/afs/infn.it/roma1/user/xxxx/Crono/POLICLINICO/RAW"

2024/03/28 10:15:47 DEBUG : sftp://cobbs@xxx.xxx.x.xx:22/Crono/POLICLINICO/RAW: Relative path joined with current directory to get absolute path "/afs/infn.it/roma1/user/xxxx/Crono/POLICLINICO/RAW"
2024/03/28 10:15:47 DEBUG : sftp://cobbs@xxx.xxx.x.xx:22/Crono/POLICLINICO/RAW: Using root directory "/afs/infn.it/roma1/user/xxxx/Crono/POLICLINICO/RAW"
2

I think if you want to use an ssh tunnel then I'd expect to see two ssh commands in there

--sftp-ssh "ssh cobbs@xxx.xxx.x.xx ssh xxxxx@login.roma1.infn.it"

Remember that when you use --sftp-ssh rclone ignores the user/host/pass in the config file so you need to set up this bit of the connection yourself in the --sftp-ssh command.

3

Hi ncw,
What I would like to do is to replicate this command.

rsync -a -e 'ssh -J xxxx@login.roma1.infn.it' prova.rtf cobbs@xxx.xxx.x.xx:/volume2/Crono/POLICLINICO/RAW

I tried as you suggest but it doesn't seem to work.

2024/03/28 11:57:10 DEBUG : rclone: Version "v1.66.0" starting with parameters ["C:\\Users\\mescs\\Rclone\\rclone.exe" "-vv" "--sftp-ssh" "ssh cobbs@xxx.xxx.x.xx ssh xxxxx@login.roma1.infn.it" "--sftp-path-override" "/volume2/Crono/POLICLINICO/RAW" "copy" "D:\\prova.rtf" "Synology:Crono/POLICLINICO/RAW"]
2024/03/28 11:57:10 DEBUG : Creating backend with remote "D:\\prova.rtf"
2024/03/28 11:57:10 DEBUG : Using config file from "C:\\Users\\mescs\\AppData\\Roaming\\rclone\\rclone.conf"
2024/03/28 11:57:10 DEBUG : fs cache: adding new entry for parent of "D:\\prova.rtf", "//?/D:/"
2024/03/28 11:57:10 DEBUG : Creating backend with remote "Synology:Crono/POLICLINICO/RAW"
2024/03/28 11:57:10 DEBUG : Synology: detected overridden config - adding "{e4l_I}" suffix to name
2024/03/28 11:57:10 NOTICE: Synology{e4l_I}: --sftp-ssh is in use - ignoring user/host/port from config - set in the parameters to --sftp-ssh (remove them from the config to silence this warning)
2024/03/28 11:57:10 DEBUG : sftp://cobbs@xxx.xxx.x.xx:22/Crono/POLICLINICO/RAW: ssh external: creating additional session
2024/03/28 11:57:10 DEBUG : sftp://cobbs@xxx.xxx.x.xx:22/Crono/POLICLINICO/RAW: ssh external: running: ssh cobbs@xxx.xxx.x.xx ssh xxxx@login.roma1.infn.it -s sftp
2024/03/28 11:57:31 DEBUG : pacer: low level retry 1/10 (error couldn't initialise SFTP: error receiving version packet from server: server unexpectedly closed connection: unexpected EOF)
4

I think you want this in your --sftp-ssh

--sftp-ssh "ssh -J xxxx@login.roma1.infn.it prova.rtf cobbs@xxx.xxx.x.xx"

I gave that a quick test and I think it should work.

5

Hi ncw,
thank you for your reply.

this command seems to work.

.\rclone.exe -vv --sftp-ssh "ssh -J xxxx@login.roma1.infn.it cobbs@xxx.xxx.x.xx" --sftp-path-override /volume2/Crono/POLICLINICO/RAW copy D:\prova.rtf Synology:Crono/POLICLINICO/RAW

the only problem now is that it ask for the cobbs@xxx.xxx.x.xx password
It is not possible to overcome this issue?

6

Rclone can't supply a username/password to the --sftp-ssh command - you'll need to arrange that separately.

Are you using an ssh-agent? If so you need to have agent-forwarding I think

--sftp-ssh "ssh -A  -J xxxx@login.roma1.infn.it cobbs@xxx.xxx.x.xx"

Or you could supply a key with the -i flag to ssh

--sftp-ssh "ssh -i ssh-identity-file -J xxxx@login.roma1.infn.it cobbs@xxx.xxx.x.xx"
7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.