--ignore-checksum not respected on 'rclone check...'

Suspected Bug
1

What is the problem you are having with rclone?

When comparing existing files from local source to OneDrive, hash check is useless for Office files because of OneDrive's file conversion. I wanted to validate name/date only, and I used the following:

rclone check <src> <dest> --ignore-checksum --ignore-size

However, I still get a mismatch error
ERROR: QuickXorHash differ

What is your rclone version (output from rclone version)

v1.50.2

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows Server 2012 R2 64-bit

Which cloud storage system are you using? (eg Google Drive)

OneDrive (Sharepoint)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone check <src> <dest> --ignore-checksum --ignore-size

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

Hard to copy from the output on Win2012, but basically I see:

DEBUG: <filename>: QuickXorHash = <hash string> (Local file system at <path>)
DEBUG: <filename>: QuickXorHash = <hash string> (OneDrive root <path>)
ERROR: <filename> QuickXorHash differ

Am I missing something, or is this flag not respected in rclone check?

2

hello and weclome to the fourm,
this is not a bug.

here are some links for you to read

1.https://rclone.org/docs/#ignore-checksum
"Normally rclone will check that the checksums of transferred files match, and give an error “corrupted on transfer” if they don’t. You can use this option to skip that check. "
so that flag is used when transferring files.

  1. https://rclone.org/commands/rclone_check/
    "If you supply the --size-only flag, it will only compare the sizes not the hashes as well. Use this for a quick check."
3

Unfortunately, I can't use --size-only either, because OneDrive converts the files to an internal format, which alters the file size. For Office docs, the only reliable parameter to match is the date.

I assumed that check uses the same underlying implementation as the checking phase of copy, but apparently that isn't true.

Is there any way to validate on DATE ONLY?

4

  1. when testing use this flag -n, --dry-run Do a trial run with no permanent changes

  2. another link:
    https://rclone.org/onedrive/#troubleshooting

1 Like
5

While I would still consider my original issue a bug of sorts - it at least bears mentioning that this flag does nothing on check - this workaround is very helpful.

For clarity for other readers, I ended up running:
rclone copy src:path dest:path --ignore-size --ignore-checksum --dry-run

Actually, I ended up doing --ignore-existing because some weird things happened with the backup method we used and some files appear to have had their touch dates randomly changed, along with some other weirdness. But @asdffdsa's suggestion resolved my original question.

6

glad we got it working.

keep in mind that rclone is a free command line, very reliable tool
as such, in the beginning there is a learning curve.
that is why there is --dry-run and this forum...

i am sure that over time you will learn of many other great items like --backup-dir, rclone mount and encrypted folders.

but did you know you can donate?
i assure you that donation does not go to me.
https://rclone.org/donate/

7

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.