Running copy/sync commands cause SharePoint Files to get nixed -- itemNotFound: The upload session was not found

an_engelsen · January 12, 2022, 4:15am

Summary:

I am trying to sync files from my local machine to SharePoint Online. Each time I run the sync command the file copy fails (on the 1st attempt) with the following error message: itemNotFound: The upload session was not found.

Context:

I am running RCLONE on Windows Server 2012 R2.
The remote destination type is: Microsoft OneDrive
The remote destination actually points to a SharePoint Online document library.
I did not edit/modify the "advanced configuration" settings while using the interactive configuration utility.

rclone version

rclone v1.57.0

os/version: Microsoft Windows Server 2012 R2 Standard Service Pack 1 (64 bit)
os/type: windows
os/arch: amd64
go/version: go1.17.2
go/linking: dynamic
go/tags: cmount

Storage system:

Microsoft OneDrive (SharePoint)

RCLONE Command (that produces the error message)

rclone.exe sync --ignore-checksum --ignore-size "E:\SharepointSync\reporting" SharePoint_Reporting_Design:

The rclone.conf file:

[SharePoint_Reporting_Design]
type = onedrive
token = {"access_token":"aVERYLongToken"}
drive_id = driveID_Was_Here
drive_type = documentLibrary

A log from the command with the `-vv` flag

λ cd "C:\Program Files\RCLONE\"

# First sync attempt shows "upload session was not found" ERROR message.
C:\Program Files\RCLONE
λ rclone.exe sync --ignore-size "E:\SharepointSync\reporting\Design" SharePoint_Reporting_Design:
2022/01/11 22:13:57 NOTICE: Visual Selling/File1.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/11 22:13:57 ERROR : Visual Selling/File1.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/11 22:13:57 ERROR : One drive root '': not deleting files as there were IO errors
2022/01/11 22:13:57 ERROR : One drive root '': not deleting directories as there were IO errors
2022/01/11 22:13:57 ERROR : Attempt 1/3 failed with 1 errors and: itemNotFound: The upload session was not found
2022/01/11 22:14:00 ERROR : Visual Selling/File1.xlsx: corrupted on transfer: quickxor hash differ "1bdde707f595f4932765983090e73cb62d95584f" vs "d25cb13443f0028da9c2acd8c01a92005370a4c2"
2022/01/11 22:14:00 ERROR : One drive root '': not deleting files as there were IO errors
2022/01/11 22:14:02 ERROR : One drive root '': not deleting directories as there were IO errors
2022/01/11 22:14:02 ERROR : Attempt 2/3 failed with 1 errors and: corrupted on transfer: quickxor hash differ "1bdde707f595f4932765983090e73cb62d95584f" vs "d25cb13443f0028da9c2acd8c01a92005370a4c2"
2022/01/11 22:14:05 ERROR : Visual Selling/File1.xlsx: corrupted on transfer: quickxor hash differ "1bdde707f595f4932765983090e73cb62d95584f" vs "b5025a64e1e9813edbc2e69640790eae6e1f8c53"
2022/01/11 22:14:06 ERROR : One drive root '': not deleting files as there were IO errors
2022/01/11 22:14:07 ERROR : One drive root '': not deleting directories as there were IO errors
2022/01/11 22:14:07 ERROR : Attempt 3/3 failed with 1 errors and: corrupted on transfer: quickxor hash differ "1bdde707f595f4932765983090e73cb62d95584f" vs "b5025a64e1e9813edbc2e69640790eae6e1f8c53"
2022/01/11 22:14:07 Failed to sync: corrupted on transfer: quickxor hash differ "1bdde707f595f4932765983090e73cb62d95584f" vs "b5025a64e1e9813edbc2e69640790eae6e1f8c53"

# Second sync attempt does NOT mention "upload session was not found".
C:\Program Files\RCLONE
λ rclone.exe sync --ignore-size "E:\SharepointSync\reporting\Design" SharePoint_Reporting_Design:
2022/01/11 22:16:37 ERROR : Visual Selling/File1.xlsx: corrupted on transfer: quickxor hash differ "1bdde707f595f4932765983090e73cb62d95584f" vs "0360cbebfc1a432b9fd7238da39c46c4a927f630"
2022/01/11 22:16:38 ERROR : One drive root '': not deleting files as there were IO errors
2022/01/11 22:16:39 ERROR : One drive root '': not deleting directories as there were IO errors
2022/01/11 22:16:39 ERROR : Attempt 1/3 failed with 1 errors and: corrupted on transfer: quickxor hash differ "1bdde707f595f4932765983090e73cb62d95584f" vs "0360cbebfc1a432b9fd7238da39c46c4a927f630"
2022/01/11 22:16:43 ERROR : Visual Selling/File1.xlsx: corrupted on transfer: quickxor hash differ "1bdde707f595f4932765983090e73cb62d95584f" vs "03faafc7f8eaaff6bdc62957052d8b18fa555ca5"
2022/01/11 22:16:43 ERROR : One drive root '': not deleting files as there were IO errors
2022/01/11 22:16:44 ERROR : One drive root '': not deleting directories as there were IO errors
2022/01/11 22:16:44 ERROR : Attempt 2/3 failed with 1 errors and: corrupted on transfer: quickxor hash differ "1bdde707f595f4932765983090e73cb62d95584f" vs "03faafc7f8eaaff6bdc62957052d8b18fa555ca5"
2022/01/11 22:16:48 ERROR : Visual Selling/File1.xlsx: corrupted on transfer: quickxor hash differ "1bdde707f595f4932765983090e73cb62d95584f" vs "7e46324dcabb50f94a37fdfdd9e71c632fa5f9f6"
2022/01/11 22:16:48 ERROR : One drive root '': not deleting files as there were IO errors
2022/01/11 22:16:49 ERROR : One drive root '': not deleting directories as there were IO errors
2022/01/11 22:16:49 ERROR : Attempt 3/3 failed with 1 errors and: corrupted on transfer: quickxor hash differ "1bdde707f595f4932765983090e73cb62d95584f" vs "7e46324dcabb50f94a37fdfdd9e71c632fa5f9f6"
2022/01/11 22:16:49 Failed to sync: corrupted on transfer: quickxor hash differ "1bdde707f595f4932765983090e73cb62d95584f" vs "7e46324dcabb50f94a37fdfdd9e71c632fa5f9f6"

# Copy command succeeds because I am using the "ignore-checksum" flag. Plus, the file got nixed after I ran the "sync" command.
C:\Program Files\RCLONE
λ rclone.exe copy --ignore-checksum --ignore-size "E:\SharepointSync\reporting\Design\Visual Selling\File1.xlsx" SharePoint_Reporting_Design:"Visual Selling"

# I modified the .xlsx file. However, I don't understand why the sync command is compaining about a change to the MD5 hash.
# Should the sync command "care" about a file that changes?
# Or should it instead just identify changed files & update them on the remote destination?
C:\Program Files\RCLONE
λ rclone.exe sync --ignore-size "E:\SharepointSync\reporting\Design\Visual Selling\File1.xlsx" SharePoint_Reporting_Design:"Visual Selling"
2022/01/11 22:27:12 NOTICE: File1.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/11 22:27:12 ERROR : File1.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/11 22:27:12 ERROR : Attempt 1/3 failed with 1 errors and: itemNotFound: The upload session was not found
2022/01/11 22:27:15 ERROR : File1.xlsx: corrupted on transfer: quickxor hash differ "e85d0df9da0471aab7ce6923db83e5b28bdf5fec" vs "8a10a979661c1bd3a4ad053ddd5979c75c9b1132"
2022/01/11 22:27:16 ERROR : Attempt 2/3 failed with 1 errors and: corrupted on transfer: quickxor hash differ "e85d0df9da0471aab7ce6923db83e5b28bdf5fec" vs "8a10a979661c1bd3a4ad053ddd5979c75c9b1132"
2022/01/11 22:27:20 ERROR : File1.xlsx: corrupted on transfer: quickxor hash differ "e85d0df9da0471aab7ce6923db83e5b28bdf5fec" vs "03de57a8fa96a9d96c701034df0d00bc5134e755"
2022/01/11 22:27:21 ERROR : Attempt 3/3 failed with 1 errors and: corrupted on transfer: quickxor hash differ "e85d0df9da0471aab7ce6923db83e5b28bdf5fec" vs "03de57a8fa96a9d96c701034df0d00bc5134e755"
2022/01/11 22:27:22 Failed to sync: corrupted on transfer: quickxor hash differ "e85d0df9da0471aab7ce6923db83e5b28bdf5fec" vs "03de57a8fa96a9d96c701034df0d00bc5134e755"

# Sync command succeeds when I include the --ignore-checksum flag.
C:\Program Files\RCLONE
λ rclone.exe sync --ignore-checksum --ignore-size "E:\SharepointSync\reporting\Design\Visual Selling\File1.xlsx" SharePoint_Reporting_Design:"Visual Selling"

# I made another edit to the .xlsx file.
# The sync command fails. (upload session was not found)
# Succeeds on the 2nd attempt.
# I continued to make edits to the file. Each sync command failed on the 1st attempt. But succeeded on the 2nd attempt.
C:\Program Files\RCLONE
λ rclone.exe sync --ignore-checksum --ignore-size "E:\SharepointSync\reporting\Design\Visual Selling\File1.xlsx" SharePoint_Reporting_Design:"Visual Selling"
2022/01/11 22:32:36 NOTICE: File1.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/11 22:32:36 ERROR : File1.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/11 22:32:36 ERROR : Attempt 1/3 failed with 1 errors and: itemNotFound: The upload session was not found
2022/01/11 22:32:40 ERROR : Attempt 2/3 succeeded

Ole · January 12, 2022, 2:40pm

Hi Michael,

Do you see the same issue if you try this:

cd "C:\Program Files\RCLONE\"
mkdir testfolder
copy readme.txt testfolder
rclone.exe sync testfolder SharePoint_Reporting_Design:testfolder -vv

if so, then please share the output.

an_engelsen · January 12, 2022, 4:52pm

No. The message about itemNotFound: The upload session was not found does not show-up when I try syncing the test folder that you asked me to create. I modifying the contents of the readme file & performed another RCLONE sync. That job also succeeded!

λ cd "C:\Program Files\RCLONE\"

C:\Program Files\RCLONE
λ mkdir TEMP

C:\Program Files\RCLONE
λ copy readme.txt TEMP
        1 file(s) copied.

C:\Program Files\RCLONE
λ rclone.exe sync TEMP SharePoint_Reporting_Design:TEMP -vv
2022/01/12 10:48:22 DEBUG : rclone: Version "v1.57.0" starting with parameters ["rclone.exe" "sync" "TEMP" "SharePoint_Reporting_Design:TEMP" "-vv"]
2022/01/12 10:48:22 DEBUG : Creating backend with remote "TEMP"
2022/01/12 10:48:22 DEBUG : Using config file from "C:\\Users\\adm_e\\AppData\\Roaming\\rclone\\rclone.conf"
2022/01/12 10:48:22 DEBUG : fs cache: renaming cache item "TEMP" to be canonical "//?/C:/Program Files/RCLONE/TEMP"
2022/01/12 10:48:22 DEBUG : Creating backend with remote "SharePoint_Reporting_Design:TEMP"
2022/01/12 10:48:24 DEBUG : One drive root 'TEMP': Waiting for checks to finish
2022/01/12 10:48:24 DEBUG : One drive root 'TEMP': Waiting for transfers to finish
2022/01/12 10:48:24 DEBUG : README.txt: Starting multipart upload
2022/01/12 10:48:25 DEBUG : README.txt: Uploading segment 0/1372473 size 1372473
2022/01/12 10:48:27 DEBUG : README.txt: quickxor = 315e22a57ba9b3a81fca32bd81889658296930de OK
2022/01/12 10:48:27 INFO  : README.txt: Copied (new)
2022/01/12 10:48:27 DEBUG : Waiting for deletions to finish
2022/01/12 10:48:27 INFO  :
Transferred:        1.309 MiB / 1.309 MiB, 100%, 446.764 KiB/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:         5.5s

2022/01/12 10:48:27 DEBUG : 7 go routines active

# Running the sync command again (after modifying the readme file).
C:\Program Files\RCLONE
λ rclone.exe sync TEMP SharePoint_Reporting_Design:TEMP -vv
2022/01/12 11:50:36 DEBUG : rclone: Version "v1.57.0" starting with parameters ["rclone.exe" "sync" "TEMP" "SharePoint_Reporting_Design:TEMP" "-vv"]
2022/01/12 11:50:36 DEBUG : Creating backend with remote "TEMP"
2022/01/12 11:50:36 DEBUG : Using config file from "C:\\Users\\adm_engelsen\\AppData\\Roaming\\rclone\\rclone.conf"
2022/01/12 11:50:36 DEBUG : fs cache: renaming cache item "TEMP" to be canonical "//?/C:/Program Files/RCLONE/TEMP"
2022/01/12 11:50:36 DEBUG : Creating backend with remote "SharePoint_Reporting_Design:TEMP"
2022/01/12 11:50:38 DEBUG : One drive root 'TEMP': Waiting for checks to finish
2022/01/12 11:50:38 DEBUG : README.txt: Sizes differ (src 1370239 vs dst 1372473)
2022/01/12 11:50:38 DEBUG : One drive root 'TEMP': Waiting for transfers to finish
2022/01/12 11:50:38 DEBUG : README.txt: Starting multipart upload
2022/01/12 11:50:39 DEBUG : README.txt: Uploading segment 0/1370239 size 1370239
2022/01/12 11:50:42 DEBUG : README.txt: quickxor = a8730821c9158dfac353e95a37b7b6dfbfe3b796 OK
2022/01/12 11:50:42 INFO  : README.txt: Copied (replaced existing)
2022/01/12 11:50:42 DEBUG : Waiting for deletions to finish
2022/01/12 11:50:42 INFO  :
Transferred:        1.307 MiB / 1.307 MiB, 100%, 448.764 KiB/s, ETA 0s
Checks:                 1 / 1, 100%
Transferred:            1 / 1, 100%
Elapsed time:         6.0s

2022/01/12 11:50:42 DEBUG : 7 go routines active

an_engelsen · January 12, 2022, 5:04pm

I have 10 .xlsx files located inside of a folder called Bookings. Nine of the files failed to sync because the The upload session was not found.

Here is the verbose output that I get when I try to perform the sync. The relevant information can be found on the lines that start with the word "ERROR".


C:\Program Files\RCLONE
λ "C:\Program Files\RCLONE\rclone.exe" sync --ignore-checksum --ignore-size "E:\SharepointSync\reporting\Wholesale Sales" SharePoint_Reporting_Wholesale_Sales: -vv
2022/01/12 10:57:12 DEBUG : rclone: Version "v1.57.0" starting with parameters ["C:\\Program Files\\RCLONE\\rclone.exe" "sync" "--ignore-checksum" "--ignore-size" "E:\\SharepointSync\\reporting\\Wholesale Sales" "SharePoint_Reporting_Wholesale_Sales:" "-vv"]
2022/01/12 10:57:12 DEBUG : Creating backend with remote "E:\\SharepointSync\\reporting\\Wholesale Sales"
2022/01/12 10:57:12 DEBUG : Using config file from "C:\\Users\\adm_engelsen\\AppData\\Roaming\\rclone\\rclone.conf"
2022/01/12 10:57:12 DEBUG : fs cache: renaming cache item "E:\\SharepointSync\\reporting\\Wholesale Sales" to be canonical "//?/E:/SharepointSync/reporting/Wholesale Sales"
2022/01/12 10:57:12 DEBUG : Creating backend with remote "SharePoint_Reporting_Wholesale_Sales:"
2022/01/12 10:57:12 DEBUG : One drive root '': Token expired but no uploads in progress - doing nothing
2022/01/12 10:57:12 DEBUG : SharePoint_Reporting_Wholesale_Sales: Loaded invalid token from config file - ignoring
2022/01/12 10:57:12 DEBUG : Saving config "token" in section "SharePoint_Reporting_Wholesale_Sales" of the config file
2022/01/12 10:57:12 DEBUG : Keeping previous permissions for config file: -rw-rw-rw-
2022/01/12 10:57:12 DEBUG : SharePoint_Reporting_Wholesale_Sales: Saved new token in config file
2022/01/12 10:57:14 DEBUG : Bookings/IssueFile09.xlsx: Modification times differ by 1396h22m20.0172473s: 2021-11-15 05:33:42.9827527 -0500 EST, 2022-01-12 14:56:03 +0000 UTC
2022/01/12 10:57:14 DEBUG : Bookings/IssueFile04.xlsx: Modification times differ by 4h28m46.6850406s: 2022-01-12 05:27:14.3149594 -0500 EST, 2022-01-12 14:56:01 +0000 UTC
2022/01/12 10:57:14 DEBUG : Bookings/IssueFile01.xlsx: Modification times differ by 4h28m39.5037945s: 2022-01-12 05:27:17.4962055 -0500 EST, 2022-01-12 14:55:57 +0000 UTC
2022/01/12 10:57:14 DEBUG : Bookings/IssueFile06.xlsx: Modification times differ by 4h28m50.9291823s: 2022-01-12 05:27:15.0708177 -0500 EST, 2022-01-12 14:56:06 +0000 UTC
2022/01/12 10:57:14 DEBUG : Bookings/IssueFile03.xlsx: Modification times differ by 4h28m42.4078651s: 2022-01-12 05:27:16.5921349 -0500 EST, 2022-01-12 14:55:59 +0000 UTC
2022/01/12 10:57:14 DEBUG : Bookings/IssueFile08.xlsx: Modification times differ by 1365h16m34.6157152s: 2021-11-16 12:39:39.3842848 -0500 EST, 2022-01-12 14:56:14 +0000 UTC
2022/01/12 10:57:14 DEBUG : Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: Modification times differ by 4h28m57.081717s: 2022-01-12 05:27:15.918283 -0500 EST, 2022-01-12 14:56:13 +0000 UTC 2022/01/12 10:57:14 DEBUG : Bookings/IssueFile02.xlsx: Modification times differ by 4h28m41.1107642s: 2022-01-12 05:27:17.8892358 -0500 EST, 2022-01-12 14:55:59 +0000 UTC
2022/01/12 10:57:14 DEBUG : One drive root '': Waiting for checks to finish
2022/01/12 10:57:14 DEBUG : Bookings/IssueFile02.xlsx: quickxor = fa39553daedec2c9a40dad031ea77a904cd4fdc2 (Local file system at //?/E:/SharepointSync/reporting/Wholesale Sales)
2022/01/12 10:57:15 DEBUG : Bookings/IssueFile02.xlsx: quickxor = 02c24c83cd06ad4b9c109c6c7ee7fccf5beaf003 (One drive root '')
2022/01/12 10:57:15 DEBUG : Bookings/IssueFile02.xlsx: quickxor differ
2022/01/12 10:57:15 DEBUG : Bookings/IssueFile05.xlsx: Modification times differ by 4h28m45.8623001s: 2022-01-12 05:27:16.1376999 -0500 EST, 2022-01-12 14:56:02 +0000 UTC
2022/01/12 10:57:15 DEBUG : Bookings/IssueFile04.xlsx: quickxor = 943e3d01ff939ddb1c2ffee2088d0d96099ce50a (Local file system at //?/E:/SharepointSync/reporting/Wholesale Sales)
2022/01/12 10:57:15 DEBUG : Bookings/IssueFile04.xlsx: quickxor = ba3eb34642be960ee32e03008704a5ec0a701b7d (One drive root '')
2022/01/12 10:57:15 DEBUG : Bookings/IssueFile04.xlsx: quickxor differ
2022/01/12 10:57:15 DEBUG : Bookings/IssueFile05.xlsx: quickxor = dfee030765a14f7ad79122af6150bba1fe1c6b5f (Local file system at //?/E:/SharepointSync/reporting/Wholesale Sales)
2022/01/12 10:57:15 DEBUG : Bookings/IssueFile05.xlsx: quickxor = 1d33b15515ee1da04b1c2ff2d900d204f4c6307d (One drive root '')
2022/01/12 10:57:15 DEBUG : Bookings/IssueFile05.xlsx: quickxor differ
2022/01/12 10:57:15 DEBUG : Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: quickxor = 6ce2ef5ca1d6a5c74d15d23636b1bda68d6f140c (Local file system at //?/E:/SharepointSync/reporting/Wholesale Sales)
2022/01/12 10:57:15 DEBUG : Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: quickxor = de3361263b1e7298de9eb79548b7c4d59225be18 (One drive root '')
2022/01/12 10:57:15 DEBUG : Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: quickxor differ
2022/01/12 10:57:15 DEBUG : Bookings/IssueFile08.xlsx: quickxor = 9487a7f8e16ac0b4e625e1f1c171d82467ddfc70 (Local file system at //?/E:/SharepointSync/reporting/Wholesale Sales)
2022/01/12 10:57:15 DEBUG : Bookings/IssueFile08.xlsx: quickxor = 178a9598e9515374f6f03472a44e5bf6a63bd5d1 (One drive root '')
2022/01/12 10:57:15 DEBUG : Bookings/IssueFile08.xlsx: quickxor differ
2022/01/12 10:57:15 DEBUG : Bookings/IssueFile07.xlsx: quickxor = 82fe07a7a39cb38eaa3bc4c224ab659addcbd72e (Local file system at //?/E:/SharepointSync/reporting/Wholesale Sales)
2022/01/12 10:57:15 DEBUG : Bookings/IssueFile07.xlsx: quickxor = be92d6fd7ed2e47d44251c9e18e117dbb1c654db (One drive root '')
2022/01/12 10:57:15 DEBUG : Bookings/IssueFile07.xlsx: quickxor differ
2022/01/12 10:57:15 DEBUG : One drive root '': Waiting for transfers to finish
2022/01/12 10:57:16 DEBUG : Bookings/IssueFile01.xlsx: Uploading segment 0/14274845 size 10485760
2022/01/12 10:57:17 DEBUG : Bookings/IssueFile02.xlsx: Uploading segment 0/15612667 size 10485760
2022/01/12 10:57:17 DEBUG : Bookings/IssueFile04.xlsx: Uploading segment 0/22789110 size 10485760
2022/01/12 10:57:17 DEBUG : Bookings/IssueFile03.xlsx: Uploading segment 0/18375930 size 10485760
2022/01/12 10:57:19 DEBUG : Bookings/IssueFile01.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:57:19 DEBUG : Bookings/IssueFile02.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:57:19 DEBUG : Bookings/IssueFile03.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:57:19 DEBUG : Bookings/IssueFile04.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:57:19 NOTICE: Bookings/IssueFile01.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:57:19 ERROR : Bookings/IssueFile01.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:57:19 DEBUG : Bookings/IssueFile09.xlsx: Starting multipart upload
2022/01/12 10:57:19 NOTICE: Bookings/IssueFile02.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:57:19 ERROR : Bookings/IssueFile02.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:57:19 DEBUG : Bookings/IssueFile06.xlsx: Starting multipart upload
2022/01/12 10:57:19 NOTICE: Bookings/IssueFile03.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:57:19 ERROR : Bookings/IssueFile03.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:57:19 DEBUG : Bookings/IssueFile05.xlsx: Starting multipart upload
2022/01/12 10:57:19 NOTICE: Bookings/IssueFile04.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:57:19 ERROR : Bookings/IssueFile04.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:57:19 DEBUG : Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: Starting multipart upload
2022/01/12 10:57:20 DEBUG : Bookings/IssueFile09.xlsx: Uploading segment 0/27698690 size 10485760
2022/01/12 10:57:21 DEBUG : Bookings/IssueFile05.xlsx: Uploading segment 0/7617229 size 7617229
2022/01/12 10:57:21 DEBUG : Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: Uploading segment 0/33305770 size 10485760
2022/01/12 10:57:21 DEBUG : Bookings/IssueFile06.xlsx: Uploading segment 0/26076139 size 10485760
2022/01/12 10:57:22 DEBUG : Bookings/IssueFile09.xlsx: Uploading segment 10485760/27698690 size 10485760
2022/01/12 10:57:23 DEBUG : Bookings/IssueFile05.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:57:24 DEBUG : Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:57:24 NOTICE: Bookings/IssueFile05.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:57:24 DEBUG : Bookings/IssueFile06.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:57:24 ERROR : Bookings/IssueFile05.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:57:24 DEBUG : Bookings/IssueFile08.xlsx: Starting multipart upload
2022/01/12 10:57:25 NOTICE: Bookings/IssueFile06.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:57:25 NOTICE: Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:57:25 DEBUG : Bookings/IssueFile09.xlsx: Uploading segment 20971520/27698690 size 6727170
2022/01/12 10:57:26 ERROR : Bookings/IssueFile06.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:57:26 ERROR : Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:57:26 DEBUG : Bookings/IssueFile07.xlsx: Starting multipart upload
2022/01/12 10:57:27 DEBUG : Bookings/IssueFile08.xlsx: Uploading segment 0/32403915 size 10485760
2022/01/12 10:57:27 DEBUG : Bookings/IssueFile07.xlsx: Uploading segment 0/20714037 size 10485760
2022/01/12 10:57:29 DEBUG : Bookings/IssueFile08.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:57:29 DEBUG : Bookings/IssueFile07.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:57:30 NOTICE: Bookings/IssueFile07.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:57:30 ERROR : Bookings/IssueFile07.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:57:30 NOTICE: Bookings/IssueFile08.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:57:30 ERROR : Bookings/IssueFile08.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:57:33 INFO  : Bookings/IssueFile09.xlsx: Copied (replaced existing)
2022/01/12 10:57:33 ERROR : One drive root '': not deleting files as there were IO errors
2022/01/12 10:57:33 ERROR : One drive root '': not deleting directories as there were IO errors
2022/01/12 10:57:33 ERROR : Attempt 1/3 failed with 9 errors and: itemNotFound: The upload session was not found
2022/01/12 10:57:38 DEBUG : Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: quickxor = 6ce2ef5ca1d6a5c74d15d23636b1bda68d6f140c (Local file system at //?/E:/SharepointSync/reporting/Wholesale Sales)
2022/01/12 10:57:38 DEBUG : Bookings/IssueFile06.xlsx: quickxor = b7fe3daef1ed2a1b4cd57cdd9d96e7602a2c8ddc (Local file system at //?/E:/SharepointSync/reporting/Wholesale Sales)
2022/01/12 10:57:38 DEBUG : Bookings/IssueFile07.xlsx: quickxor = be92d6fd7ed2e47d44251c9e18e117dbb1c654db (One drive root '')
2022/01/12 10:57:38 DEBUG : Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: quickxor = de3361263b1e7298de9eb79548b7c4d59225be18 (One drive root '')
2022/01/12 10:57:38 DEBUG : Bookings/IssueFile06.xlsx: quickxor = 29e20ce2a1dc9451641a0023cd5084268ad75af4 (One drive root '')
2022/01/12 10:57:38 DEBUG : Bookings/IssueFile06.xlsx: quickxor differ
2022/01/12 10:57:38 DEBUG : Bookings/IssueFile07.xlsx: quickxor differ
2022/01/12 10:57:38 DEBUG : Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: quickxor differ
2022/01/12 10:57:38 DEBUG : One drive root '': Waiting for transfers to finish
2022/01/12 10:57:38 DEBUG : Bookings/IssueFile04.xlsx: Uploading segment 0/22789110 size 10485760
2022/01/12 10:57:38 DEBUG : Bookings/IssueFile01.xlsx: Uploading segment 10485760/14274845 size 3789085
2022/01/12 10:57:39 DEBUG : Bookings/IssueFile09.xlsx: Uploading segment 0/27698690 size 10485760
2022/01/12 10:57:40 DEBUG : Bookings/IssueFile09.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:57:41 NOTICE: Bookings/IssueFile09.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:57:41 INFO  : Bookings/IssueFile05.xlsx: Copied (replaced existing)
2022/01/12 10:57:41 DEBUG : Bookings/IssueFile04.xlsx: Uploading segment 10485760/22789110 size 10485760
2022/01/12 10:57:42 ERROR : Bookings/IssueFile09.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:58:10 ERROR : One drive root '': not deleting files as there were IO errors
2022/01/12 10:58:10 ERROR : One drive root '': not deleting directories as there were IO errors
2022/01/12 10:58:10 ERROR : Attempt 2/3 failed with 1 errors and: itemNotFound: The upload session was not found
2022/01/12 10:58:11 DEBUG : One drive root '': Waiting for transfers to finish
2022/01/12 10:58:13 DEBUG : Bookings/IssueFile01.xlsx: Uploading segment 0/14274845 size 10485760
2022/01/12 10:58:13 INFO  :
Transferred:      305.993 MiB / 514.723 MiB, 59%, 3.155 MiB/s, ETA 1m6s
Checks:                30 / 30, 100%
Transferred:           10 / 20, 50%
Elapsed time:       1m1.1s
Transferring:
 * Bookings/Bookings with…ustomer Info F'22.xlsx:  0% /13.614Mi, 0/s, -
 * Bookings/Bookings with…ustomer Info F'21.xlsx:  0% /21.733Mi, 0/s, -
 * Bookings/Bookings with…ization Info S'21.xlsx:  0% /7.264Mi, 0/s, -
 * Bookings/Bookings with…ustomer Info S'22.xlsx:  0% /17.525Mi, 0/s, -

2022/01/12 10:58:13 DEBUG : Bookings/IssueFile05.xlsx: Uploading segment 0/7617229 size 7617229
2022/01/12 10:58:13 DEBUG : Bookings/IssueFile03.xlsx: Uploading segment 0/18375930 size 10485760
2022/01/12 10:58:13 DEBUG : Bookings/IssueFile04.xlsx: Uploading segment 0/22789110 size 10485760
2022/01/12 10:58:15 DEBUG : Bookings/IssueFile05.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:58:15 NOTICE: Bookings/IssueFile05.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:58:15 ERROR : Bookings/IssueFile05.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:58:15 DEBUG : Bookings/IssueFile06.xlsx: Starting multipart upload
2022/01/12 10:58:15 DEBUG : Bookings/IssueFile01.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:58:16 NOTICE: Bookings/IssueFile01.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:58:16 DEBUG : Bookings/IssueFile04.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:58:17 ERROR : Bookings/IssueFile01.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:58:16 DEBUG : Bookings/IssueFile03.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:58:17 DEBUG : Bookings/IssueFile02.xlsx: Starting multipart upload
2022/01/12 10:58:17 NOTICE: Bookings/IssueFile03.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:58:17 ERROR : Bookings/IssueFile03.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:58:17 DEBUG : Bookings/IssueFile09.xlsx: Starting multipart upload
2022/01/12 10:58:17 NOTICE: Bookings/IssueFile04.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:58:17 ERROR : Bookings/IssueFile04.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:58:17 DEBUG : Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: Starting multipart upload
2022/01/12 10:58:17 DEBUG : Bookings/IssueFile06.xlsx: Uploading segment 0/26076139 size 10485760
2022/01/12 10:58:18 DEBUG : Bookings/IssueFile09.xlsx: Uploading segment 0/27698690 size 10485760
2022/01/12 10:58:19 DEBUG : Bookings/IssueFile02.xlsx: Uploading segment 0/15612667 size 10485760
2022/01/12 10:58:20 DEBUG : Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: Uploading segment 0/33305770 size 10485760
2022/01/12 10:58:20 DEBUG : Bookings/IssueFile06.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:58:20 DEBUG : Bookings/IssueFile02.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:58:21 NOTICE: Bookings/IssueFile02.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:58:21 ERROR : Bookings/IssueFile02.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:58:21 DEBUG : Bookings/IssueFile08.xlsx: Starting multipart upload
2022/01/12 10:58:21 NOTICE: Bookings/IssueFile06.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:58:21 ERROR : Bookings/IssueFile06.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:58:21 DEBUG : Bookings/IssueFile07.xlsx: Starting multipart upload
2022/01/12 10:58:21 DEBUG : Bookings/IssueFile09.xlsx: Uploading segment 10485760/27698690 size 10485760
2022/01/12 10:58:21 DEBUG : Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:58:22 NOTICE: Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:58:23 ERROR : Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:58:23 DEBUG : Bookings/IssueFile09.xlsx: Uploading segment 20971520/27698690 size 6727170
2022/01/12 10:58:23 DEBUG : Bookings/IssueFile07.xlsx: Uploading segment 0/20714037 size 10485760
2022/01/12 10:58:23 DEBUG : Bookings/IssueFile08.xlsx: Uploading segment 0/32403915 size 10485760
2022/01/12 10:58:25 DEBUG : Bookings/IssueFile07.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:58:25 DEBUG : Bookings/IssueFile08.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 10:58:26 NOTICE: Bookings/IssueFile07.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:58:26 ERROR : Bookings/IssueFile07.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:58:26 NOTICE: Bookings/IssueFile08.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 10:58:26 ERROR : Bookings/IssueFile08.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 10:58:30 INFO  : Bookings/IssueFile09.xlsx: Copied (replaced existing)
2022/01/12 10:58:30 ERROR : One drive root '': not deleting files as there were IO errors
2022/01/12 10:58:30 ERROR : One drive root '': not deleting directories as there were IO errors
2022/01/12 10:58:30 ERROR : Attempt 3/3 failed with 9 errors and: itemNotFound: The upload session was not found
2022/01/12 10:58:30 INFO  :
Transferred:      419.673 MiB / 419.673 MiB, 100%, 4.930 MiB/s, ETA 0s
Errors:                 9 (retrying may help)
Checks:                30 / 30, 100%
Transferred:           11 / 11, 100%
Elapsed time:      1m18.5s

2022/01/12 10:58:30 DEBUG : 7 go routines active
2022/01/12 10:58:30 Failed to sync with 9 errors: last error was: itemNotFound: The upload session was not found

Ole · January 12, 2022, 7:37pm

OK, then let's try a two step sync of a single file to isolate the issue.

What do you see if you try this:

cd "C:\Program Files\RCLONE\"
mkdir "E:\SharepointSync\reporting\Wholesale Sales\testfolder2"
copy  "E:\SharepointSync\reporting\Wholesale Sales\Bookings\IssueFile01.xlsx" E:\SharepointSync\reporting\Wholesale Sales\testfolder2"
mkdir testfolder2
rclone sync "E:\SharepointSync\reporting\Wholesale Sales\testfolder2" testfolder2 --ignore-times -vv
rclone sync testfolder2 SharePoint_Reporting_Wholesale_Sales: --ignore-times -vv

an_engelsen · January 12, 2022, 11:50pm

The first sync job succeeded. (local volume --> local volume)

C:\Program Files\RCLONE
λ rclone.exe sync "E:\SharepointSync\reporting\Wholesale Sales\testfolder2" testfolder2 --ignore-times -vv
2022/01/12 18:22:12 DEBUG : rclone: Version "v1.57.0" starting with parameters ["rclone.exe" "sync" "E:\\SharepointSync\\reporting\\Wholesale Sales\\testfolder2" "testfolder2" "--ignore-times" "-vv"]
2022/01/12 18:22:12 DEBUG : Creating backend with remote "E:\\SharepointSync\\reporting\\Wholesale Sales\\testfolder2"
2022/01/12 18:22:12 DEBUG : Using config file from "C:\\Users\\adm_engelsen\\AppData\\Roaming\\rclone\\rclone.conf"
2022/01/12 18:22:12 DEBUG : fs cache: renaming cache item "E:\\SharepointSync\\reporting\\Wholesale Sales\\testfolder2" to be canonical "//?/E:/SharepointSync/reporting/Wholesale Sales/testfolder2"
2022/01/12 18:22:12 DEBUG : Creating backend with remote "testfolder2"
2022/01/12 18:22:12 DEBUG : fs cache: renaming cache item "testfolder2" to be canonical "//?/C:/Program Files/RCLONE/testfolder2"
2022/01/12 18:22:12 DEBUG : Local file system at //?/C:/Program Files/RCLONE/testfolder2: Waiting for checks to finish
2022/01/12 18:22:12 DEBUG : Local file system at //?/C:/Program Files/RCLONE/testfolder2: Waiting for transfers to finish
2022/01/12 18:22:12 DEBUG : Bookings with Material and Customer with Customization Info S'21.xlsx: md5 = 3457afa3ddbe358ae97fa6a8f4e7456a OK
2022/01/12 18:22:12 INFO  : Bookings with Material and Customer with Customization Info S'21.xlsx: Copied (new)
2022/01/12 18:22:12 DEBUG : Waiting for deletions to finish
2022/01/12 18:22:12 INFO  :
Transferred:        7.264 MiB / 7.264 MiB, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         0.1s

2022/01/12 18:22:12 DEBUG : 3 go routines active

The 2nd sync job failed because the file sizes differed on transfer.

Failed to sync: corrupted on transfer: sizes differ 7617229 vs 7624530

When I add the --ignore-size flag, I was told that the hashes did not match. (Which is very odd.)

Failed to sync: corrupted on transfer: quickxor hash differ "dfee030765a14f7ad79122af6150bba1fe1c6b5f" vs "ba8674f47885eebe48a86d526ba3ef11ae65227b"

Adding both the --ignore-size and --ignore-checksum flags allowed the file transfer to succeed.

C:\Program Files\RCLONE
λ rclone sync testfolder2 SharePoint_Reporting_Wholesale_Sales: --ignore-times --ignore-size --ignore-checksum -vv
2022/01/12 18:32:46 DEBUG : rclone: Version "v1.57.0" starting with parameters ["rclone" "sync" "testfolder2" "SharePoint_Reporting_Wholesale_Sales:" "--ignore-times" "--ignore-size" "--ignore-checksum" "-vv"]
2022/01/12 18:32:46 DEBUG : Creating backend with remote "testfolder2"
2022/01/12 18:32:46 DEBUG : Using config file from "C:\\Users\\adm_engelsen\\AppData\\Roaming\\rclone\\rclone.conf"
2022/01/12 18:32:46 DEBUG : fs cache: renaming cache item "testfolder2" to be canonical "//?/C:/Program Files/RCLONE/testfolder2"
2022/01/12 18:32:46 DEBUG : Creating backend with remote "SharePoint_Reporting_Wholesale_Sales:"
2022/01/12 18:32:48 DEBUG : Bookings with Material and Customer with Customization Info S'21.xlsx: Starting multipart upload
2022/01/12 18:32:48 DEBUG : One drive root '': Waiting for checks to finish
2022/01/12 18:32:49 DEBUG : One drive root '': Waiting for transfers to finish
2022/01/12 18:32:49 DEBUG : Bookings with Material and Customer with Customization Info S'21.xlsx: Uploading segment 0/7617229 size 7617229
2022/01/12 18:32:58 INFO  : Bookings with Material and Customer with Customization Info S'21.xlsx: Copied (new)
2022/01/12 18:32:58 DEBUG : Waiting for deletions to finish
2022/01/12 18:32:58 INFO  : Bookings/Bookings with Material and Customer Info F'22.xlsx: Deleted
2022/01/12 18:32:58 INFO  : Bookings/Bookings with Material and Customer Info S'21.xlsx: Deleted
2022/01/12 18:32:58 INFO  : Bookings/Bookings with Material and Customer with Customization Info S'21.xlsx: Deleted
2022/01/12 18:32:58 INFO  : Bookings/Bookings with Material and Customer with Customization Info F'21.xlsx: Deleted
2022/01/12 18:32:59 INFO  : Bookings/Bookings with Material and Customer Info S'22.xlsx: Deleted
2022/01/12 18:32:59 INFO  : Bookings/Bookings with Material and Customer Info F'20.xlsx: Deleted
2022/01/12 18:32:59 INFO  : Bookings/Bookings with Material and Customer Info F'21.xlsx: Deleted
2022/01/12 18:32:59 INFO  : Bookings/Bookings with Material and Customer with Customization Info F'20.xlsx: Deleted
2022/01/12 18:32:59 INFO  : Bookings/Bookings with Material and Customer with Customization Info F'22.xlsx: Deleted
2022/01/12 18:32:59 INFO  : Bookings/Bookings with Material and Customer with Customization Info S'22.xlsx: Deleted
2022/01/12 18:33:00 INFO  : Bookings: Removing directory
2022/01/12 18:33:00 DEBUG : One drive root '': deleted 1 directories
2022/01/12 18:33:00 INFO  :
Transferred:        7.264 MiB / 7.264 MiB, 100%, 618.771 KiB/s, ETA 0s
Checks:                10 / 10, 100%
Deleted:               10 (files), 1 (dirs)
Transferred:            1 / 1, 100%
Elapsed time:        15.2s

2022/01/12 18:33:00 DEBUG : 13 go routines active

I edited the .xlsx file and ran another sync job (still ignoring the size, checksum, and time). The file transfer succeeded, but I did see output that said the upload session was not found.

C:\Program Files\RCLONE
λ rclone sync testfolder2 SharePoint_Reporting_Wholesale_Sales: --ignore-times --ignore-size --ignore-checksum -vv
2022/01/12 18:41:04 DEBUG : rclone: Version "v1.57.0" starting with parameters ["rclone" "sync" "testfolder2" "SharePoint_Reporting_Wholesale_Sales:" "--ignore-times" "--ignore-size" "--ignore-checksum" "-vv"]
2022/01/12 18:41:05 DEBUG : Creating backend with remote "testfolder2"
2022/01/12 18:41:05 DEBUG : Using config file from "C:\\Users\\adm_engelsen\\AppData\\Roaming\\rclone\\rclone.conf"
2022/01/12 18:41:05 DEBUG : fs cache: renaming cache item "testfolder2" to be canonical "//?/C:/Program Files/RCLONE/testfolder2"
2022/01/12 18:41:05 DEBUG : Creating backend with remote "SharePoint_Reporting_Wholesale_Sales:"
2022/01/12 18:41:06 DEBUG : Bookings with Material and Customer with Customization Info S'21.xlsx: Transferring unconditionally as --ignore-times is in use
2022/01/12 18:41:06 DEBUG : One drive root '': Waiting for checks to finish
2022/01/12 18:41:06 DEBUG : One drive root '': Waiting for transfers to finish
2022/01/12 18:41:06 DEBUG : Bookings with Material and Customer with Customization Info S'21.xlsx: Starting multipart upload
2022/01/12 18:41:08 DEBUG : Bookings with Material and Customer with Customization Info S'21.xlsx: Uploading segment 0/7939438 size 7939438
2022/01/12 18:41:09 DEBUG : Bookings with Material and Customer with Customization Info S'21.xlsx: Cancelling multipart upload: itemNotFound: The upload session was not found
2022/01/12 18:41:09 NOTICE: Bookings with Material and Customer with Customization Info S'21.xlsx: Failed to cancel multipart upload: itemNotFound: The upload session was not found (upload failed due to: itemNotFound: The upload session was not found)
2022/01/12 18:41:09 ERROR : Bookings with Material and Customer with Customization Info S'21.xlsx: Failed to copy: itemNotFound: The upload session was not found
2022/01/12 18:41:09 ERROR : One drive root '': not deleting files as there were IO errors
2022/01/12 18:41:09 ERROR : One drive root '': not deleting directories as there were IO errors
2022/01/12 18:41:09 ERROR : Attempt 1/3 failed with 1 errors and: itemNotFound: The upload session was not found
2022/01/12 18:41:09 DEBUG : Bookings with Material and Customer with Customization Info S'21.xlsx: Transferring unconditionally as --ignore-times is in use
2022/01/12 18:41:09 DEBUG : One drive root '': Waiting for checks to finish
2022/01/12 18:41:10 DEBUG : One drive root '': Waiting for transfers to finish
2022/01/12 18:41:10 DEBUG : Bookings with Material and Customer with Customization Info S'21.xlsx: Starting multipart upload
2022/01/12 18:41:10 DEBUG : Bookings with Material and Customer with Customization Info S'21.xlsx: Uploading segment 0/7939438 size 7939438
2022/01/12 18:41:13 INFO  : Bookings with Material and Customer with Customization Info S'21.xlsx: Copied (replaced existing)
2022/01/12 18:41:13 DEBUG : Waiting for deletions to finish
2022/01/12 18:41:13 ERROR : Attempt 2/3 succeeded
2022/01/12 18:41:13 INFO  :
Transferred:       15.143 MiB / 15.143 MiB, 100%, 2.514 MiB/s, ETA 0s
Checks:                 2 / 2, 100%
Transferred:            1 / 1, 100%
Elapsed time:         8.8s

2022/01/12 18:41:13 DEBUG : 7 go routines active

Here's what I think:

The file transfer will succeed when RCLONE is told to sync a "small batches" of files. (In this example we updating just 1 file.) However, when RCLONE is asked to sync a folder with 10+ flles, that is when things start to fail miserably. What do you think, @Ole ?

Ole · January 13, 2022, 8:51am

My best guess is that you are experiencing these two known issues:
https://rclone.org/onedrive/#unexpected-file-size-hash-differences-on-sharepoint
https://rclone.org/onedrive/#replacing-deleting-existing-files-on-sharepoint-gets-item-not-found

The last of these probably has a somewhat random nature. I guess it could be caused by a meta data race condition while SharePoint is silently modifying the file.

Try copying app. 10 issue files to testfolder2 on C: and then check if you can make this error:

rclone sync testfolder2 SharePoint_Reporting_Wholesale_Sales:testfolder2 --backup-dir SharePoint_Reporting_Wholesale_Sales:testfolder2backup --ignore-checksum --ignore-size --ignore-times -v

Repeat up to 5 times if you see no error; I added --ignore-times to force updates while testing.

asdffdsa · January 13, 2022, 1:59pm

perhaps using one of these flags might help?
https://rclone.org/docs/#delete-before-during-after

Ole · January 13, 2022, 3:14pm

I don’t think so, the issues we see above are related to creation and update (of Office files).

an_engelsen · January 14, 2022, 3:19pm

I appreciate all of the helpful advice. I created a scheduled task that performs a RCLONE sync job (once every day). I have been sending the results of the sync job to a log file.

@echo off
title RCLONE_sync

set "currentDate=%date:~-4,4%%date:~-7,2%%date:~-10,2%_"
set "currentTime=%time:~0,2%_%time:~3,2%"
set "resultsDoc=ResultsOn_%currentDate%%currentTime%.txt"

"C:\Program Files\RCLONE\rclone.exe" sync "E:\reporting\Wholesale Sales" SharePoint_Reporting_Wholesale_Sales: --ignore-checksum --ignore-size --log-file="C:\RCLONE_Logs\WHOLESALE_%resultsDoc%" --log-level=ERROR

EXIT 0

Based on the log files, the RCLONE sync job is still failing to copy the files to SharePoint on the 1st attempt.

Reason: Failed to copy: itemNotFound: The upload session was not found

However, the sync job has consistently been succeeding on attempt 2 (out of three).

I would love to see the sync job succeed on the 1st attempt. However, at this point in time I think this is the closest we will be able to get to a "working solution".

Ole · January 14, 2022, 4:31pm

Thanks, glad to hear that you found a workable solution.

Your observations strengthen my suspicion of a race condition (random behavior). The somewhat acceptable pattern you are seeing now may therefore change abruptly by a minor change in data or computer/network timings. It may change to sync in first attempt and then suddenly change to be unable to sync no matter how many attempts; time will show.

I therefore recommend you somehow continue to monitor the success of your sync and have a plan B ready. (E.g. --backup-dir)

system · February 13, 2022, 4:31pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.