OneDrive Size/Hash differs

I'm attempting to copy the contents of a OneDrive library to a SharePoint library using the following command.
rclone copy OneDrive:\ SharePoint:\

--dry-run shows everything correctly, no error messages.

My config is very simplistic.and connects without issue.

[OneDrive]
type = onedrive
token = *removed*
drive_id = *removed*
drive_type = business

[SharePoint]
type = onedrive
token = *removed*
drive_id = *removed*
drive_type = documentLibrary

However when I try to copy or sync it seems to error on every MS Office file (.docx, .doc, .xlsx, etc..) with the same basic error about sizes that differ. PDF and TXT copy without issue.

ERROR : Billing/Monthly Paying Clients.docx: corrupted on transfer: sizes differ 14244 vs 20069

And if I try --ignore-size or --ignore-time I get hash differs

ERROR : Billing/Monthly Paying Clients.docx: corrupted on transfer: QuickXorHash hash differ "e15e26fddd6b748ae65f863b5e024b2a7c53e852" vs "e73f88ddbb715d17e436fab1dc8d8e9503ea7ec8"

Can you verify you are on the latest version and run the same command with -vv and share the output?

Sure thing, I created a Test folder with one test.docx file in it. It almost looks like the file size is changing on every access... I wonder if this has something to do with OneDrive versioning?

c:\rclone>rclone -V
rclone v1.49.2
- os/arch: windows/amd64
- go version: go1.12.3

c:\rclone>rclone -vv copy OneDrive:\Test SharePoint:\Test
2019/09/17 17:03:03 DEBUG : rclone: Version "v1.49.2" starting with parameters ["rclone" "-vv" "copy" "OneDrive:\\Test" "SharePoint:\\Test"]
2019/09/17 17:03:03 DEBUG : Using config file from "C:\\Users\\myuser\\.config\\rclone\\rclone.conf"
2019/09/17 17:03:04 INFO  : One drive root 'Test': Waiting for checks to finish
2019/09/17 17:03:04 INFO  : One drive root 'Test': Waiting for transfers to finish
2019/09/17 17:03:06 DEBUG : Test.docx: Starting multipart upload
2019/09/17 17:03:07 DEBUG : Test.docx: Uploading segment 0/11086 size 11086
2019/09/17 17:03:09 ERROR : Test.docx: corrupted on transfer: sizes differ 11086 vs 17279
2019/09/17 17:03:09 INFO  : Test.docx: Removing failed copy
2019/09/17 17:03:09 ERROR : Attempt 1/3 failed with 3 errors and: corrupted on transfer: sizes differ 11086 vs 17279
2019/09/17 17:03:09 INFO  : One drive root 'Test': Waiting for checks to finish
2019/09/17 17:03:09 INFO  : One drive root 'Test': Waiting for transfers to finish
2019/09/17 17:03:12 DEBUG : Test.docx: Starting multipart upload
2019/09/17 17:03:13 DEBUG : Test.docx: Uploading segment 0/11086 size 11086
2019/09/17 17:03:15 ERROR : Test.docx: corrupted on transfer: sizes differ 11086 vs 17283
2019/09/17 17:03:15 INFO  : Test.docx: Removing failed copy
2019/09/17 17:03:16 ERROR : Attempt 2/3 failed with 3 errors and: corrupted on transfer: sizes differ 11086 vs 17283
2019/09/17 17:03:16 INFO  : One drive root 'Test': Waiting for checks to finish
2019/09/17 17:03:16 INFO  : One drive root 'Test': Waiting for transfers to finish
2019/09/17 17:03:16 DEBUG : Test.docx: Starting multipart upload
2019/09/17 17:03:17 DEBUG : Test.docx: Uploading segment 0/11086 size 11086
2019/09/17 17:03:19 ERROR : Test.docx: corrupted on transfer: sizes differ 11086 vs 17281
2019/09/17 17:03:19 INFO  : Test.docx: Removing failed copy
2019/09/17 17:03:19 ERROR : Attempt 3/3 failed with 3 errors and: corrupted on transfer: sizes differ 11086 vs 17281
2019/09/17 17:03:19 Failed to copy with 3 errors: last error was: corrupted on transfer: sizes differ 11086 vs 17281

I think what happens is that onedrive converts the .doc into an internal format so you can edit it then when you copy it back it converts that internal format back into a .doc file.

You can probably work around this with --ignore-size --ignore-checksum.

This isn't ideal though!

If you are doing this sync regularly then I'd probably do one with --exclude "*.{doc,xls,...}" and one with --include "*.{doc,xls,...}" --ignore-size --ignore-checksum

1 Like

It looks like the --ignore-size --ignore-checksum does the trick.

Do you know if sync will work with these switches after my initial copy? According to the synopsis on https://rclone.org/commands/rclone_sync/ it works by "testing by size and modification time or MD5SUM", and i'll essentially be removing the size and MD5SUM checks so I'm not sure how reliable it will be.

Thank you!

That is why I suggested doing two syncs, one for the normal files and one for the docs. It will use modtime so it should detect changes, but it isn't as reliable as it could be.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.