Windows sha1sum via SFTP

Suspected Bug
1

What is the problem you are having with rclone?

I am using the official OpenSSH that is now part of windows (link below). This works well for file transfer but hash checks do not seem to work.

I have tested using both sha1sum and md5sums.

https://www.adaic.org/resources/add_content/standards/sha1sum.exe
--sftp-sha1sum-command "sha1sum"

http://www.pc-tools.net/files/win32/freeware/md5sums-1.2.zip
--sftp-md5sum-command "md5sums -u"

The first issue seems to be the path being different to sftp and i got around this using --sftp-path-override "C:\test folder".

Looking at the console output the issue seems to be the way rclone handles the escape characters and backslashes.

I think the best way forward would most likely just be an OS detection on SSH connect or a optional flag for Windows.

What is your rclone version (output from rclone version)

v1.57.0-beta.5801.bb11803f1

Which OS you are using and how many bits (e.g. Windows 7, 64 bit)

Client - Ubuntu 21.10
Server - Windows Server 2019

Which cloud storage system are you using? (e.g. Google Drive)

SFTP

The command you were trying to run (e.g. rclone copy /tmp remote:tmp)

rclone sha1sum "windows:/C:/test folder/example1/" --sftp-sha1sum-command "sha1sum" -vv

A log from the command with the -vv flag (e.g. output from rclone -vv copy /tmp remote:tmp)

2021/10/26 18:08:39 DEBUG : rclone: Version "v1.57.0-beta.5801.bb11803f1" starting with parameters ["rclone" "sha1sum" "windows:/C:/test folder/example1/" "--sftp-sha1sum-command" "sha1sum" "-vv"]
2021/10/26 18:08:39 DEBUG : Creating backend with remote "windows:/C:/test folder/example1/"
2021/10/26 18:08:39 DEBUG : Using config file from "/home/user/.config/rclone/rclone.conf"
2021/10/26 18:08:39 DEBUG : windows: detected overridden config - adding "{h5EBD}" suffix to name
2021/10/26 18:08:40 DEBUG : sftp://*****@*****//C:/test folder/example1/: New connection *****:39518->*****:22 to "SSH-2.0-OpenSSH_for_Windows_8.1"
2021/10/26 18:08:40 DEBUG : fs cache: renaming cache item "windows:/C:/test folder/example1/" to be canonical "windows{h5EBD}:/C:/test folder/example1/"
2021/10/26 18:08:40 DEBUG : sftp cmd = /C:/test\ folder/example1/file1.txt
2021/10/26 18:08:40 DEBUG : file1.txt: Failed to calculate sha1 hash: Process exited with status 1 (can't open `/C:/test\': Invalid argument)
                                          file1.txt
2021/10/26 18:08:40 DEBUG : 11 go routines active
2

hello and welcome to the forum,

not sure this is a bug, as rclone might not yet support windows.

  • in the mean time, as a possible workaround
    rclone can act as a sftp server and supports hashes.
    rclone serve sftp

  • i have not yet tried openssh server on windows, as for many years i have been using bitvise.
    think it is time for me to test it out......

3

Based on troubleshooting done so far on github (Windows sha1sum via SFTP · Issue #5758 · rclone/rclone · GitHub), the issue seems related to spaces.

Please let me know if you have the same folder space issue using bitvise?

4

did not know there was an open github issue.

with bitvise server, the primay use has been to protect rdp sessions, have not tried sftp file transfer yet

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.