Best way to verify checksum when using copy command and save bandwidth

[StarkWiz]

What is the problem you are having with rclone?

I am connected to my PC in a different location over VPN. I want to use rclone to selectively download files from the PC. Considering this connectivity is over VPN. I would like checksum verification as well to ensure that the file copied over locally is intact.
Source is a windows 10 fileshare/smb and destination is a local windows 10 machine.
I tried using the --checksum option but I noticed that it uses extra bandwidth to create checksum from the source.
I tried going through different topics here and use the hashum option to create a hash file in the remote pc but I am not able to use that hash file on the local pc.

--check-file option doesn't seem to apply for the copy command.

So, I would like to know what would be the best way here to have some sort of remote side checksum being used.
I am ok with generating checksum files manually or through script if required on the remote pc. If there is any way to use the same on the local pc.

I also noticed that there is serve option available with rclone but not sure if any of those support providing a server-side checksum that rclone on my local pc can use.

What is your rclone version (output from rclone version)

I am using the latest version that is 1.56.0

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10 64-bit

Which cloud storage system are you using? (eg Google Drive)

None, Destination is Windows 10 Local NTFS Filesystem and Source is Windows 10 SMB Shared Location

The command you were trying to run (eg rclone copy /tmp remote:tmp)

I have shortened the source url here to hide personal information by using ... here

rclone.exe copy "\\192.168.10.5\...\DCIM\100GOPRO\GH010488.MP4" .  -P --checksum -vv --check-first -check-file "\\192.168.10.5\...\DCIM\100GOPRO\100GOPRO.sha1"

The rclone config contents with secrets removed.

Not using any config file.

A log from the command with the -vv flag

Error: unknown flag: --check-file

[asdffdsa (jojothehumanmonkey)]

hello and welcome to the forum,

as per the documentation, the correct flag would be --checkfile
tho it would seem, based on rclone flag naming conventions it should be --check-file

@StarkWiz

file copied over locally is intact.

rclone sync/copy/move does that as part of the transfer process.
if you use debug output by adding -vv to the command, you would see this

DEBUG : kdbx.20210815.153940.7z: md5 = 644fb4e16b1572cbe3ddf77123ba5565 OK
DEBUG : kdbx.20210815.153940.7z: OK

@StarkWiz

there is serve option available with rclone but not sure if any of those support providing a server-side checksum that rclone on my local pc can use.

rclone serve sftp will compare checksums for each file transferred.

[StarkWiz]

@asdffdsa Thanks for the quick response.
I actually tried now --checkfile as well but still it gives error below, when using it with the copy command.

Error: unknown flag: --checkfile

[asdffdsa (jojothehumanmonkey)]

--checkfile works with rclone hashsum, not rclone copy.

[StarkWiz]

@asdffdsa

rclone serve sftp will compare checksums for each file transferred.

Thanks sftp does the job of verifying. Is there any way to force sha1sum verification instead of md5 ? I did notice that --checksum parameter is required to use checksum else it just compares size and datetime.

@asdffdsa

--checkfile works with rclone hashsum, not rclone copy.

Yes, right.

[asdffdsa (jojothehumanmonkey)]

@StarkWiz

force sha1sum

well, i do not know, but it seems that if you ask for a sha1 checksum, then rclone serve sftp will return that.

curious, for your use-case, what is the practical difference between sha1 and md5?

[StarkWiz]

I think SHA1 is more reliable but thinking about my actual requirement, it probably wouldn't matter.

Thanks a lot for your help @asdffdsa

I'll work further on setting up rclone serve sftp to start automatically.

[asdffdsa (jojothehumanmonkey)]

@StarkWiz

I think SHA1 is more reliable

that is news to me but whichever checksum you want, with rclone serve sftp it is rclone that calculates the checksum.

---

@StarkWiz

rclone serve sftp to start automatically

i use windows task scheduler, has a nice GUI and can start a command / batch file at startup.

[StarkWiz]

@asdffdsa

that is news to me but whichever checksum you want, with rclone serve sftp it is rclone that calculates the checksum.

Do you mean to say I can select SHA1 as well ? How ?

Also, thanks for the task scheduler suggestion.

[asdffdsa (jojothehumanmonkey)]

i tried and failed with a bunch of ideas and tricks to force rclone to use sha1.
in the end, rclone forces the use of md5.

i ran a rclone serve sftp on the local folder.
i created a sftp remote pointing to that sftp server
i played around with flags and commands.

with the rclone sftp backend config file, whenever i run a command:
if comment out md5sum_command = md5sum and sha1sum_command = sha1sum

• rclone will add md5sum_command = md5sum
  *rclone will NOT add sha1sum_command = sha1sum
• only when using a command such as rclone sha1sum, did rclone add sha1sum_command = sha1sum

if i comment out md5sum_command = md5sum and do not command out sha1sum_command = sha1sum
rclone will still not use sha and rclone will un-comment md5sum_command = md5sum

i also tried this, hoping rclone would use sha1 but still rclone used md5
rclone copy file.txt shatest:test --sftp-sha1sum-command=sha1sum -vv

to force sha1 over md5, i think a new flag would be needed such as
--hash-type=sha1

[StarkWiz]

@asdffdsa Thank you for checking this for me. Yes, indeed it seems hash-type option would be required. I guess I'll leave it just with the sftp setup now and md5sum.