--sftp-ssh and --sftp-path-override do not work together

Help and Support
1

What is the problem you are having with rclone?

when I try to copy files with an SSH tunnel.
But the command doesn't work.
Is it possible that --sftp-ssh and --sftp-path-override can't work together?

Run the command 'rclone version' and share the full output of the command.

rclone v1.65.2
- os/version: Microsoft Windows 11 Pro 23H2 (64 bit)
- os/kernel: 10.0.22631.3155 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.21.6
- go/linking: static
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

I'm trying to copy data to a Synology NAS

The command you were trying to run

./rclone.exe copy D:\TEST Crono:Crono/RAW --sftp-ssh "ssh user@server" --sftp-path-override /volume2/Crono/RAW

Please run 'rclone config redacted' and share the full output.

[Crono]
type = sftp
host = XXX
user = XXX
pass = XXX
shell_type = unix
md5sum_command = md5sum
sha1sum_command = sha1sum

A log from the command that you were trying to run with the -vv flag

2024/03/01 13:40:05 DEBUG : rclone: Version "v1.65.2" starting with parameters ["C:\\Users\\mescs\\Rclone\\rclone.exe" "-vv" "copy" "D:\\TEST" "Crono:Crono/RAW" "--sftp-ssh" "ssh parisile@login.roma1.infn.it" "--sftp-path-override" "/volume2/Crono/RAW"]
2024/03/01 13:40:05 DEBUG : Creating backend with remote "D:\\TEST"
2024/03/01 13:40:05 DEBUG : Using config file from "C:\\Users\\mescs\\AppData\\Roaming\\rclone\\rclone.conf"
2024/03/01 13:40:05 DEBUG : fs cache: renaming cache item "D:\\TEST" to be canonical "//?/D:/TEST"
2024/03/01 13:40:05 DEBUG : Creating backend with remote "Crono:Crono/RAW"
2024/03/01 13:40:05 DEBUG : Crono: detected overridden config - adding "{7uZFm}" suffix to name
2024/03/01 13:40:05 NOTICE: Crono{7uZFm}: --sftp-ssh is in use - ignoring user/host/port from config - set in the parameters to --sftp-ssh (remove them from the config to silence this warning)
2024/03/01 13:40:05 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: ssh external: creating additional session
2024/03/01 13:40:05 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: ssh external: running: ssh parisile@login.roma1.infn.it -s sftp
parisile@login.roma1.infn.it's password:
2024/03/01 13:40:14 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: Shell type "unix" from config
2024/03/01 13:40:14 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: Relative path resolved to "/afs/infn.it/roma1/user/parisile/Crono/RAW"
2024/03/01 13:40:14 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: Using root directory "/afs/infn.it/roma1/user/parisile/Crono/RAW"
2024/03/01 13:40:14 DEBUG : fs cache: renaming cache item "Crono:Crono/RAW" to be canonical "Crono{7uZFm}:Crono/RAW"
2024/03/01 13:40:14 DEBUG : test.rtf: Need to transfer - File not found at Destination
2024/03/01 13:40:14 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: Waiting for checks to finish
2024/03/01 13:40:14 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: Waiting for transfers to finish
2024/03/01 13:40:14 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: Shell path redirected to "/volume2/Crono/RAW/test.rtf.xariyet4.partial" with option path_override
2024/03/01 13:40:14 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: ssh external: creating additional session
2024/03/01 13:40:14 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: Running remote command: md5sum /volume2/Crono/RAW/test.rtf.xariyet4.partial
2024/03/01 13:40:14 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: ssh external: running: ssh parisile@login.roma1.infn.it "md5sum /volume2/Crono/RAW/test.rtf.xariyet4.partial"
parisile@login.roma1.infn.it's password:
2024/03/01 13:40:18 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: ssh external: command exited with error: exit status 1
2024/03/01 13:40:18 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: ssh external: close
2024/03/01 13:40:18 ERROR : test.rtf.xariyet4.partial: Failed to calculate dst hash: failed to calculate md5 hash: failed to run "md5sum /volume2/Crono/RAW/test.rtf.xariyet4.partial": md5sum: /volume2/Crono/RAW/test.rtf.xariyet4.partial: No such file or directory: exit status 1
2024/03/01 13:40:18 ERROR : test.rtf.xariyet4.partial: corrupted on transfer: md5 hash differ "661b9fff0fc0ec8d6d1374e66ac4754e" vs ""
2024/03/01 13:40:18 INFO  : test.rtf.xariyet4.partial: Removing failed copy
2024/03/01 13:40:18 ERROR : Attempt 1/3 failed with 2 errors and: corrupted on transfer: md5 hash differ "661b9fff0fc0ec8d6d1374e66ac4754e" vs ""
2024/03/01 13:40:18 DEBUG : test.rtf: Need to transfer - File not found at Destination
2024/03/01 13:40:18 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: Waiting for checks to finish
2024/03/01 13:40:18 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: Waiting for transfers to finish
2024/03/01 13:40:18 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: Shell path redirected to "/volume2/Crono/RAW/test.rtf.kigebim7.partial" with option path_override
2024/03/01 13:40:18 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: ssh external: creating additional session
2024/03/01 13:40:18 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: Running remote command: md5sum /volume2/Crono/RAW/test.rtf.kigebim7.partial
2024/03/01 13:40:18 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: ssh external: running: ssh parisile@login.roma1.infn.it "md5sum /volume2/Crono/RAW/test.rtf.kigebim7.partial"
parisile@login.roma1.infn.it's password:
2024/03/01 13:40:23 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: ssh external: command exited with error: exit status 1
2024/03/01 13:40:23 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: ssh external: close
2024/03/01 13:40:23 ERROR : test.rtf.kigebim7.partial: Failed to calculate dst hash: failed to calculate md5 hash: failed to run "md5sum /volume2/Crono/RAW/test.rtf.kigebim7.partial": md5sum: /volume2/Crono/RAW/test.rtf.kigebim7.partial: No such file or directory: exit status 1
2024/03/01 13:40:23 ERROR : test.rtf.kigebim7.partial: corrupted on transfer: md5 hash differ "661b9fff0fc0ec8d6d1374e66ac4754e" vs ""
2024/03/01 13:40:23 INFO  : test.rtf.kigebim7.partial: Removing failed copy
2024/03/01 13:40:23 ERROR : Attempt 2/3 failed with 2 errors and: corrupted on transfer: md5 hash differ "661b9fff0fc0ec8d6d1374e66ac4754e" vs ""
2024/03/01 13:40:23 DEBUG : test.rtf: Need to transfer - File not found at Destination
2024/03/01 13:40:23 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: Waiting for checks to finish
2024/03/01 13:40:23 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: Waiting for transfers to finish
2024/03/01 13:40:23 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: Shell path redirected to "/volume2/Crono/RAW/test.rtf.hukapum6.partial" with option path_override
2024/03/01 13:40:23 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: ssh external: creating additional session
2024/03/01 13:40:23 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: Running remote command: md5sum /volume2/Crono/RAW/test.rtf.hukapum6.partial
2024/03/01 13:40:23 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: ssh external: running: ssh parisile@login.roma1.infn.it "md5sum /volume2/Crono/RAW/test.rtf.hukapum6.partial"
parisile@login.roma1.infn.it's password:
2024/03/01 13:40:33 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: ssh external: command exited with error: exit status 1
2024/03/01 13:40:33 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: ssh external: close
2024/03/01 13:40:33 ERROR : test.rtf.hukapum6.partial: Failed to calculate dst hash: failed to calculate md5 hash: failed to run "md5sum /volume2/Crono/RAW/test.rtf.hukapum6.partial": md5sum: /volume2/Crono/RAW/test.rtf.hukapum6.partial: No such file or directory: exit status 1
2024/03/01 13:40:33 ERROR : test.rtf.hukapum6.partial: corrupted on transfer: md5 hash differ "661b9fff0fc0ec8d6d1374e66ac4754e" vs ""
2024/03/01 13:40:33 INFO  : test.rtf.hukapum6.partial: Removing failed copy
2024/03/01 13:40:33 ERROR : Attempt 3/3 failed with 2 errors and: corrupted on transfer: md5 hash differ "661b9fff0fc0ec8d6d1374e66ac4754e" vs ""
2024/03/01 13:40:33 INFO  :
Transferred:      114.636 KiB / 114.636 KiB, 100%, 5.896 KiB/s, ETA 0s
Errors:                 2 (retrying may help)
Elapsed time:        28.1s

2024/03/01 13:40:33 DEBUG : 7 go routines active
2024/03/01 13:40:33 DEBUG : sftp://cobbs@141.108.4.25:22/Crono/RAW: Closing 1 unused connections
2024/03/01 13:40:33 Failed to copy with 2 errors: last error was: corrupted on transfer: md5 hash differ "661b9fff0fc0ec8d6d1374e66ac4754e" vs ""
2

is working from command line? Without asking for password?

As per docs:

Important The ssh command must log in without asking for a password so needs to be configured with keys or certificates.

3

This could be part of the problem:

I have a Synology NAS too, and for me the following config works perfectly (without --sftp-ssh):

[NAS]
type = sftp
host = XXX
user = XXX
port = XXX
pass = XXX
shell_type = unix
md5sum_command = md5sum
sha1sum_command = sha1sum
path_override = @/volume1

Note the @ prefix in path_override.

4

unfortunately due to choices of the network system administrators I cannot configure on the machine I use to do the ssh tunnel to use authorized_keys.
is there not a way to do it on rclone?
how to save the password as in the case of sftp?

5

How do I edit the configure file by hand and add path_override?

6

Yes. This is text file you can edit in any editor.

7

Where can I find it?

8

Look at your previous post with DEBUG log:) it is listed there

Or run:

$ rclone config file
9

You can not then use --sftp-ssh flag.

What do you need it for BTW?

10

Thanks!
I edited the configuration file and I added the path_override option.
I tried it and it works perfectly! (without passing the option via command).

11

the situation is a bit entangled.
I have computer A that needs to copy data to a NAS (B).
The NAS and computers are on two different networks and can only communicate via VPN.
The NAS is configured (for security reasons) in such a way that it only accepts connections from known IPs (previously configured) the VPN unfortunately changes IP addresses randomly.
I had been advised to do an SSH tunnel to a server (C) that has static IP so that it would be added as accepted IP on the NAS.
In short having this chain PC (A) -> Server (C) -> NAS (B).
Unfortunately, I discovered that (again for security reasons) on server (C) the option to put authorized_keys is disabled.
At this point I don't know what to do.

12

Configure permanent VPN tunnel between A and C maybe with only traffic to B going through it.

As you use Windows I have not much experience with this OS.

You will find plenty of info how to do it on internet - search for split tunnel

13

i will try to search.
thank you very much for the help!!

14

on windows, could use tailscale, which is what i do, no need to split tunnel, no need for middleman server.
or run tailscale on the NAS.

if you openvpn or wireguard, i have config files that do split tunnel on windows.
bascaiily, need to remove the vpn default route.
i use viscosity vpn which has a nice gui for openvpn

should not need for middleman server.

for that, i use DDNS

15

I was able to configure the authorized_keys.

I ran this command:

.\rclone.exe copy D:\prova.rtf Synology:Crono/POLICLINICO/RAW -vv --sftp-ssh "ssh xxxxx@login.roma1.infn.it" --sftp-path-override /volume2/Crono/POLICLINICO/RAW

but I have this error:

2024/03/26 12:22:04 DEBUG : rclone: Version "v1.65.2" starting with parameters ["C:\\Users\\mescs\\Rclone\\rclone.exe" "copy" "D:\\prova.rtf" "Synology:Crono/POLICLINICO/RAW" "-vv" "--sftp-ssh" "ssh xxxxx@login.roma1.infn.it" "--sftp-path-override" "/volume2/Crono/POLICLINICO/RAW"]
2024/03/26 12:22:04 DEBUG : Creating backend with remote "D:\\prova.rtf"
2024/03/26 12:22:04 DEBUG : Using config file from "C:\\Users\\mescs\\AppData\\Roaming\\rclone\\rclone.conf"
2024/03/26 12:22:04 DEBUG : fs cache: adding new entry for parent of "D:\\prova.rtf", "//?/D:/"
2024/03/26 12:22:04 DEBUG : Creating backend with remote "Synology:Crono/POLICLINICO/RAW"
2024/03/26 12:22:04 DEBUG : Synology: detected overridden config - adding "{XjOCf}" suffix to name
2024/03/26 12:22:04 NOTICE: Synology{XjOCf}: --sftp-ssh is in use - ignoring user/host/port from config - set in the parameters to --sftp-ssh (remove them from the config to silence this warning)
2024/03/26 12:22:04 DEBUG : sftp://cobbs@xxx.xxx.xxx.xxx:22/Crono/POLICLINICO/RAW: ssh external: creating additional session
2024/03/26 12:22:04 DEBUG : sftp://cobbs@xxx.xxx.xxx.xxx:22/Crono/POLICLINICO/RAW: ssh external: running: ssh xxxxx@login.roma1.infn.it -s sftp
2024/03/26 12:22:05 DEBUG : sftp://cobbs@xxx.xxx.xxx.xxx:22/Crono/POLICLINICO/RAW: Shell type "unix" from config
2024/03/26 12:22:05 DEBUG : sftp://cobbs@xxx.xxx.xxx.xxx:22/Crono/POLICLINICO/RAW: Failed to resolve path using RealPath: file does not exist
2024/03/26 12:22:05 DEBUG : sftp://cobbs@xxx.xxx.xxx.xxx:22/Crono/POLICLINICO/RAW: Relative path joined with current directory to get absolute path "/afs/infn.it/roma1/user/xxxxx/Crono/POLICLINICO/RAW"
2024/03/26 12:22:05 DEBUG : sftp://cobbs@xxx.xxx.xxx.xxx:22/Crono/POLICLINICO/RAW: Using root directory "/afs/infn.it/roma1/user/xxxxx/Crono/POLICLINICO/RAW"
2024/03/26 12:22:05 DEBUG : fs cache: renaming cache item "Synology:Crono/POLICLINICO/RAW" to be canonical "Synology{XjOCf}:Crono/POLICLINICO/RAW"
2024/03/26 12:22:05 DEBUG : prova.rtf: Need to transfer - File not found at Destination
2024/03/26 12:22:05 DEBUG : sftp://cobbs@xxx.xxx.xxx.xxx:22/Crono/POLICLINICO/RAW: Connection OK after error: permission denied
2024/03/26 12:22:05 ERROR : prova.rtf: Failed to copy: Put mkParentDir failed: mkdir "/afs/infn.it/roma1/user/parisile/Crono" failed: permission denied
2024/03/26 12:22:05 ERROR : Attempt 1//3 failed with 1 errors and: Put mkParentDir failed: mkdir "/afs/infn.it/roma1/user/xxxxx/Crono" failed: permission denied
2024/03/26 12:22:05 DEBUG : prova.rtf: Need to transfer - File not found at Destination
2024/03/26 12:22:05 DEBUG : sftp://cobbs@xxx.xxx.xxx.xxx:22/Crono/POLICLINICO/RAW: Connection OK after error: permission denied
2024/03/26 12:22:05 ERROR : prova.rtf: Failed to copy: Put mkParentDir failed: mkdir "/afs/infn.it/roma1/user/xxxxx/Crono" failed: permission denied
2024/03/26 12:22:05 ERROR : Attempt 2//3 failed with 1 errors and: Put mkParentDir failed: mkdir "/afs/infn.it/roma1/user/xxxxx/Crono" failed: permission denied
2024/03/26 12:22:05 DEBUG : prova.rtf: Need to transfer - File not found at Destination
2024/03/26 12:22:05 DEBUG : sftp://cobbs@xxx.xxx.xxx.xxx:22/Crono/POLICLINICO/RAW: Connection OK after error: permission denied
2024/03/26 12:22:05 ERROR : prova.rtf: Failed to copy: Put mkParentDir failed: mkdir "/afs/infn.it/roma1/user/xxxxx/Crono" failed: permission denied
2024/03/26 12:22:05 ERROR : Attempt 3//3 failed with 1 errors and: Put mkParentDir failed: mkdir "/afs/infn.it/roma1/user/xxxxx/Crono" failed: permission denied
2024/03/26 12:22:05 INFO  :
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.9s

2024/03/26 12:22:05 DEBUG : 7 go routines active
2024/03/26 12:22:05 DEBUG : sftp://cobbs@xxx.xxx.xxx.xxx:22/Crono/POLICLINICO/RAW: Closing 1 unused connections
2024/03/26 12:22:05 Failed to copy: Put mkParentDir failed: mkdir "/afs/infn.it/roma1/user/xxxxx/Crono" failed: permission deniedI was able to configure the authorized_keys

doesn't understand that "Crono/POLICLINICO/RAW" is on the target machine and not on the server I use as a tunnel

16

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.