Onedrive Errors

What is the problem you are having with rclone?

I am getting some error messages when uploading to my Onedrive that is attached to my Office 365 Enterprise account. We have 5 users and my account is provisioned up to 25TB of which i have used about 6TB.
The 2 major errors are:

Failed to copy: Put "FILENAME REMOVED": http2: client connection force closed via ClientConn.Close
and 
Failed to copy: corrupted on transfer: QuickXorHash crypted hash differ

Not every file I upload fails. about a week ago i would lose about 25% to the second error but over the past few days 75% or more have been failing with both errors. As an example, my last log showed that out of 900GB uploaded, 167GB were successfully uploaded without errors.

I am using a crypted onedrive remote.

In an attempt to test things, i tried uploading a file directly to onedrive via a web browser (unencrypted without rclone) and it seemed to work fine.

I also tried removing the client I'd and secret from the config file but that had no effect.

I have another 15TB to upload but I tried not uploading anything for about 2 days, thinking I may have hit some type of undocumented upload limit but after resuming, i experienced the same issue.

I have a feeling this is a onedrive issue but wanted to see if any of you had experience with this.

What is your rclone version (output from rclone version)

rclone v1.55.1
os/type: linux
os/arch: amd64
go/version: 1.16.3
go/linking: static
go/tags: none

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Centos 8

Which cloud storage system are you using? (eg Google Drive)

Onedrive via crypted remote

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync -v -P /Source /Crypted-Destination --log-file=XXXXX --stats 10m --bwlimit=off --max-transfer=off --cutoff-mode=cautious --transfers 8 --onedrive-no-versions

The rclone config contents with secrets removed.

[gdrive]
type = drive
client_id = XXXXXXXXXXXXXXX
client_secret = XXXXXXXXXXXXXXX
scope = drive
token = {"access_token":"XXXXXXXXXXXXXXX"}

[crypt]
type = crypt
remote = gdrive:Rclone/Gdrive2
filename_encryption = standard
directory_name_encryption = true
password = XXXXXXXXXXXXXXX-XXXXXXXXXXXXXXX
password2 = XXXXXXXXXXXXXXX-XXXXXXXXXXXXXXX-XXXXXXXXXXXXXXX

[onedrive]
type = onedrive
region = global
token = {"access_token":"XXXXXXXXXXXXXXX"}
drive_id = XXXXXXXXXXXXXXX
drive_type = business

[cryptonedrive]
type = crypt
remote = onedrive:
filename_encryption = standard
directory_name_encryption = true
password = XXXXXXXXXXXXXXX
password2 = XXXXXXXXXXXXXXX


A log from the command with the -vv flag

This is a log file from running the command with the -v flag. if needed i will re-run with '-vv' but that will take about a day. Let me know!

2021/06/03 01:49:29 INFO  : Danielles Music and Photos/xac: Copied (new)
2021/06/03 01:49:46 ERROR : Danielles Music and Photos/xaa: Failed to copy: corrupted on transfer: QuickXorHash crypted hash differ "526efcbd1b65c7b6fa33202cc59e6b042386fc87" vs "a293a5372e9932b61b276270b74f9f716198fd36"
2021/06/03 02:29:43 ERROR : Mimir Backups/Hyper-V VM Backups/2.5.2021/LocalCloudBackup.vhdx: Failed to copy: Put "https://hinsonventures-my.sharepoint.com/personal/jerry_reed_hinsoncompanies_com/_api/v2.0/drives/b!ksQgOXPXt0q5KsTGNXLgWS6pPDzqblhGoKy1o0_8xNX3OUCQWSQfRrKUHcpMU64d/items/016TXQIDTS2CRSFIQKBNCYPRZIIYO7UNVI/uploadSession?guid='236567c0-430e-45f1-bbfa-eb2bf273e3d1'&path='~tmp0E_a55pi7aj51ctlna0samvoquie3khjugfi8ruo5petkmtet032c0g'&overwrite=True&rename=False&dc=0&tempauth=eyJ0eXAiOiJKV1QiLCJhbGciOiJub25lIn0.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDAvaGluc29udmVudHVyZXMtbXkuc2hhcmVwb2ludC5jb21ANWJmYTE0NTktM2VkMi00ZWRjLThhY2QtZjBjM2MyNDcwNjkzIiwiaXNzIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwIiwibmJmIjoiMTYyMjY5OTM3MSIsImV4cCI6IjE2MjI3ODU3NzEiLCJlbmRwb2ludHVybCI6Imxpb1JnTktRVDJ0anltZmp0SU81dnFHVTA4WHJWTmFnbUNkVFEwWlFDcnM9IiwiZW5kcG9pbnR1cmxMZW5ndGgiOiIzNjMiLCJpc2xvb3BiYWNrIjoiVHJ1ZSIsImNpZCI6Ik1qZzRaRFEwTVdRdFlqazROUzAwWVRZd0xUZzRNVFF0WVRsaVlXVTBZMk16TVRJdyIsInZlciI6Imhhc2hlZHByb29mdG9rZW4iLCJzaXRlaWQiOiJNemt5TUdNME9USXRaRGMzTXkwMFlXSTNMV0k1TW1FdFl6UmpOak0xTnpKbE1EVTUiLCJhcHBfZGlzcGxheW5hbWUiOiJyY2xvbmUiLCJnaXZlbl9uYW1lIjoiSmVycnkiLCJmYW1pbHlfbmFtZSI6IlJlZWQiLCJhcHBpZCI6ImIxNTY2NWQ5LWVkYTYtNDA5Mi04NTM5LTBlZWMzNzZhZmQ1OSIsInRpZCI6IjViZmExNDU5LTNlZDItNGVkYy04YWNkLWYwYzNjMjQ3MDY5MyIsInVwbiI6ImplcnJ5LnJlZWRAaGluc29uY29tcGFuaWVzLmNvbSIsInB1aWQiOiIxMDAzMjAwMTQyRDRDOTZDIiwiY2FjaGVrZXkiOiIwaC5mfG1lbWJlcnNoaXB8MTAwMzIwMDE0MmQ0Yzk2Y0BsaXZlLmNvbSIsInNjcCI6Im15ZmlsZXMucmVhZCBhbGxmaWxlcy5yZWFkIG15ZmlsZXMud3JpdGUgYWxsZmlsZXMud3JpdGUgYWxsc2l0ZXMucmVhZCIsImFjcnMiOiJyMSxyMixyMyxjMSxjMixjMyIsInR0IjoiMiIsInVzZVBlcnNpc3RlbnRDb29raWUiOm51bGx9.bkVrYnlSS2Q3UmRrdkt2R295QjgrOVRURngrZ1pkVE9jMW05VFhOdkcrST0": http2: client connection force closed via ClientConn.Close
2021/06/03 02:29:50 ERROR : Mimir Backups/Hyper-V VM Backups/2.5.2021/OpenHAB.vhdx: Failed to copy: Put "https://hinsonventures-my.sharepoint.com/personal/jerry_reed_hinsoncompanies_com/_api/v2.0/drives/b!ksQgOXPXt0q5KsTGNXLgWS6pPDzqblhGoKy1o0_8xNX3OUCQWSQfRrKUHcpMU64d/items/016TXQIDTS2CRSFIQKBNCYPRZIIYO7UNVI/uploadSession?guid='4cda2592-6f67-480b-8040-ebae4ecf4234'&path='~tmp56_ne3rfs4uo9aph6pkj4hcc2bq64'&overwrite=True&rename=False&dc=0&tempauth=eyJ0eXAiOiJKV1QiLCJhbGciOiJub25lIn0.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDAvaGluc29udmVudHVyZXMtbXkuc2hhcmVwb2ludC5jb21ANWJmYTE0NTktM2VkMi00ZWRjLThhY2QtZjBjM2MyNDcwNjkzIiwiaXNzIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwIiwibmJmIjoiMTYyMjY5OTM4NiIsImV4cCI6IjE2MjI3ODU3ODYiLCJlbmRwb2ludHVybCI6ImtpRHVQOWI0d1JHYWczblVVWTZKczAvSDhqbVpFS3RnUHRualJzZ2VSenc9IiwiZW5kcG9pbnR1cmxMZW5ndGgiOiIzMzciLCJpc2xvb3BiYWNrIjoiVHJ1ZSIsImNpZCI6IlpqRmlOamxpWVRZdE16RXlNUzAwTURGaUxUbGxObUl0TUdKaE5EUXhPVEZqTVRrMyIsInZlciI6Imhhc2hlZHByb29mdG9rZW4iLCJzaXRlaWQiOiJNemt5TUdNME9USXRaRGMzTXkwMFlXSTNMV0k1TW1FdFl6UmpOak0xTnpKbE1EVTUiLCJhcHBfZGlzcGxheW5hbWUiOiJyY2xvbmUiLCJnaXZlbl9uYW1lIjoiSmVycnkiLCJmYW1pbHlfbmFtZSI6IlJlZWQiLCJhcHBpZCI6ImIxNTY2NWQ5LWVkYTYtNDA5Mi04NTM5LTBlZWMzNzZhZmQ1OSIsInRpZCI6IjViZmExNDU5LTNlZDItNGVkYy04YWNkLWYwYzNjMjQ3MDY5MyIsInVwbiI6ImplcnJ5LnJlZWRAaGluc29uY29tcGFuaWVzLmNvbSIsInB1aWQiOiIxMDAzMjAwMTQyRDRDOTZDIiwiY2FjaGVrZXkiOiIwaC5mfG1lbWJlcnNoaXB8MTAwMzIwMDE0MmQ0Yzk2Y0BsaXZlLmNvbSIsInNjcCI6Im15ZmlsZXMucmVhZCBhbGxmaWxlcy5yZWFkIG15ZmlsZXMud3JpdGUgYWxsZmlsZXMud3JpdGUgYWxsc2l0ZXMucmVhZCIsImFjcnMiOiJyMSxyMixyMyxjMSxjMixjMyIsInR0IjoiMiIsInVzZVBlcnNpc3RlbnRDb29raWUiOm51bGx9.OHZKQnBrMlk5ZDJoRm1hSEo4RjBkSC9RM3NUWGdpU0N4d1NhVVJUb0F1MD0": http2: client connection force closed via ClientConn.Close
2021/06/03 02:29:50 ERROR : Files and Records Backups/Files and Records 4.9.2021.zip: Failed to copy: Put "https://hinsonventures-my.sharepoint.com/personal/jerry_reed_hinsoncompanies_com/_api/v2.0/drives/b!ksQgOXPXt0q5KsTGNXLgWS6pPDzqblhGoKy1o0_8xNX3OUCQWSQfRrKUHcpMU64d/items/016TXQIDUHHI4KO6BDBBGJLJUFESHQBMHB/uploadSession?guid='373e78ea-de46-4582-8c48-44e89e913230'&path='~tmpA1_jbsffbi51vthfkc8v8c59u2gbbr7r3tiqp3sk49v476oteu698g0'&overwrite=True&rename=False&dc=0&tempauth=eyJ0eXAiOiJKV1QiLCJhbGciOiJub25lIn0.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDAvaGluc29udmVudHVyZXMtbXkuc2hhcmVwb2ludC5jb21ANWJmYTE0NTktM2VkMi00ZWRjLThhY2QtZjBjM2MyNDcwNjkzIiwiaXNzIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwIiwibmJmIjoiMTYyMjY3MTM0MyIsImV4cCI6IjE2MjI3NTc3NDMiLCJlbmRwb2ludHVybCI6Ijg3SXEyVEt1VDdiTkE1L2M2ZFg1aC9CTDRnTGY1aG91Zi90QitBSlFFNUk9IiwiZW5kcG9pbnR1cmxMZW5ndGgiOiIzNjMiLCJpc2xvb3BiYWNrIjoiVHJ1ZSIsImNpZCI6Ik16WXpZMlF4T0RZdFpEaGxZeTAwTURaa0xUbGxPR0l0T0RWaE16UXdNbU00Tm1JMyIsInZlciI6Imhhc2hlZHByb29mdG9rZW4iLCJzaXRlaWQiOiJNemt5TUdNME9USXRaRGMzTXkwMFlXSTNMV0k1TW1FdFl6UmpOak0xTnpKbE1EVTUiLCJhcHBfZGlzcGxheW5hbWUiOiJyY2xvbmUiLCJnaXZlbl9uYW1lIjoiSmVycnkiLCJmYW1pbHlfbmFtZSI6IlJlZWQiLCJhcHBpZCI6ImIxNTY2NWQ5LWVkYTYtNDA5Mi04NTM5LTBlZWMzNzZhZmQ1OSIsInRpZCI6IjViZmExNDU5LTNlZDItNGVkYy04YWNkLWYwYzNjMjQ3MDY5MyIsInVwbiI6ImplcnJ5LnJlZWRAaGluc29uY29tcGFuaWVzLmNvbSIsInB1aWQiOiIxMDAzMjAwMTQyRDRDOTZDIiwiY2FjaGVrZXkiOiIwaC5mfG1lbWJlcnNoaXB8MTAwMzIwMDE0MmQ0Yzk2Y0BsaXZlLmNvbSIsInNjcCI6Im15ZmlsZXMucmVhZCBhbGxmaWxlcy5yZWFkIG15ZmlsZXMud3JpdGUgYWxsZmlsZXMud3JpdGUgYWxsc2l0ZXMucmVhZCIsImFjcnMiOiJyMSxyMixyMyxjMSxjMixjMyIsInR0IjoiMiIsInVzZVBlcnNpc3RlbnRDb29raWUiOm51bGx9.dkxCRytRc2NFR2FCWitsUVk2QmxxTmdkU0hCYjN5YWF3M3JSOW5wTUpmbz0": http2: client connection force closed via ClientConn.Close
2021/06/03 02:29:50 ERROR : Files and Records Backups/Files and Records 8.25.2020.zip: Failed to copy: Put "https://hinsonventures-my.sharepoint.com/personal/jerry_reed_hinsoncompanies_com/_api/v2.0/drives/b!ksQgOXPXt0q5KsTGNXLgWS6pPDzqblhGoKy1o0_8xNX3OUCQWSQfRrKUHcpMU64d/items/016TXQIDUHHI4KO6BDBBGJLJUFESHQBMHB/uploadSession?guid='1f0daf13-3658-496c-b468-05e54e9d5083'&path='~tmpAC_nht4hlblv0egqovsive0g5t4h4be7m6361a96vr3qmu09r1l6a00'&overwrite=True&rename=False&dc=0&tempauth=eyJ0eXAiOiJKV1QiLCJhbGciOiJub25lIn0.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDAvaGluc29udmVudHVyZXMtbXkuc2hhcmVwb2ludC5jb21ANWJmYTE0NTktM2VkMi00ZWRjLThhY2QtZjBjM2MyNDcwNjkzIiwiaXNzIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwIiwibmJmIjoiMTYyMjY3MTM0MyIsImV4cCI6IjE2MjI3NTc3NDMiLCJlbmRwb2ludHVybCI6Im1YVVRCL0VkVVU3S3RLTGNsNys4Zms4QzZiQ0NMR0drMXRpbUtPK0hEaE09IiwiZW5kcG9pbnR1cmxMZW5ndGgiOiIzNjMiLCJpc2xvb3BiYWNrIjoiVHJ1ZSIsImNpZCI6Ik1ETmlaRFExT1RndFpHSmxOUzAwTURJMkxXSXhZVEV0TVdabU1qVXpNMkU0TlRaayIsInZlciI6Imhhc2hlZHByb29mdG9rZW4iLCJzaXRlaWQiOiJNemt5TUdNME9USXRaRGMzTXkwMFlXSTNMV0k1TW1FdFl6UmpOak0xTnpKbE1EVTUiLCJhcHBfZGlzcGxheW5hbWUiOiJyY2xvbmUiLCJnaXZlbl9uYW1lIjoiSmVycnkiLCJmYW1pbHlfbmFtZSI6IlJlZWQiLCJhcHBpZCI6ImIxNTY2NWQ5LWVkYTYtNDA5Mi04NTM5LTBlZWMzNzZhZmQ1OSIsInRpZCI6IjViZmExNDU5LTNlZDItNGVkYy04YWNkLWYwYzNjMjQ3MDY5MyIsInVwbiI6ImplcnJ5LnJlZWRAaGluc29uY29tcGFuaWVzLmNvbSIsInB1aWQiOiIxMDAzMjAwMTQyRDRDOTZDIiwiY2FjaGVrZXkiOiIwaC5mfG1lbWJlcnNoaXB8MTAwMzIwMDE0MmQ0Yzk2Y0BsaXZlLmNvbSIsInNjcCI6Im15ZmlsZXMucmVhZCBhbGxmaWxlcy5yZWFkIG15ZmlsZXMud3JpdGUgYWxsZmlsZXMud3JpdGUgYWxsc2l0ZXMucmVhZCIsImFjcnMiOiJyMSxyMixyMyxjMSxjMixjMyIsInR0IjoiMiIsInVzZVBlcnNpc3RlbnRDb29raWUiOm51bGx9.NzNqYWFIVGc0eUdGSFZWSU01MjZTaGxiN2ZpMXZCejd6Z25vWEJ3NzZrRT0": http2: client connection force closed via ClientConn.Close
2021/06/03 02:29:50 ERROR : Mimir Backups/Hyper-V VM Backups/1.11.2021/BlueIris.vhdx: Failed to copy: Put "https://hinsonventures-my.sharepoint.com/personal/jerry_reed_hinsoncompanies_com/_api/v2.0/drives/b!ksQgOXPXt0q5KsTGNXLgWS6pPDzqblhGoKy1o0_8xNX3OUCQWSQfRrKUHcpMU64d/items/016TXQIDV5XEDUPKIPSRBJJGBAUTWQSDUC/uploadSession?guid='5ad72120-ccfd-4b84-9275-8de7772e0b27'&path='~tmp68_q31acqk1dnioos8hk7q9ua4ga4'&overwrite=True&rename=False&dc=0&tempauth=eyJ0eXAiOiJKV1QiLCJhbGciOiJub25lIn0.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDAvaGluc29udmVudHVyZXMtbXkuc2hhcmVwb2ludC5jb21ANWJmYTE0NTktM2VkMi00ZWRjLThhY2QtZjBjM2MyNDcwNjkzIiwiaXNzIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwIiwibmJmIjoiMTYyMjY3MTM0NCIsImV4cCI6IjE2MjI3NTc3NDQiLCJlbmRwb2ludHVybCI6ImZ4Ykt0S0tTTS9oaFpPZnpVRkR2NklTTVdRZzgxdlBxN2JMUHVUMmc0bk09IiwiZW5kcG9pbnR1cmxMZW5ndGgiOiIzMzciLCJpc2xvb3BiYWNrIjoiVHJ1ZSIsImNpZCI6IllUYzVPVFUxWWpVdE1qSTRNUzAwWWpZNUxUazRZVEF0TkRjellUVXlNMll5Wm1VMCIsInZlciI6Imhhc2hlZHByb29mdG9rZW4iLCJzaXRlaWQiOiJNemt5TUdNME9USXRaRGMzTXkwMFlXSTNMV0k1TW1FdFl6UmpOak0xTnpKbE1EVTUiLCJhcHBfZGlzcGxheW5hbWUiOiJyY2xvbmUiLCJnaXZlbl9uYW1lIjoiSmVycnkiLCJmYW1pbHlfbmFtZSI6IlJlZWQiLCJhcHBpZCI6ImIxNTY2NWQ5LWVkYTYtNDA5Mi04NTM5LTBlZWMzNzZhZmQ1OSIsInRpZCI6IjViZmExNDU5LTNlZDItNGVkYy04YWNkLWYwYzNjMjQ3MDY5MyIsInVwbiI6ImplcnJ5LnJlZWRAaGluc29uY29tcGFuaWVzLmNvbSIsInB1aWQiOiIxMDAzMjAwMTQyRDRDOTZDIiwiY2FjaGVrZXkiOiIwaC5mfG1lbWJlcnNoaXB8MTAwMzIwMDE0MmQ0Yzk2Y0BsaXZlLmNvbSIsInNjcCI6Im15ZmlsZXMucmVhZCBhbGxmaWxlcy5yZWFkIG15ZmlsZXMud3JpdGUgYWxsZmlsZXMud3JpdGUgYWxsc2l0ZXMucmVhZCIsImFjcnMiOiJyMSxyMixyMyxjMSxjMixjMyIsInR0IjoiMiIsInVzZVBlcnNpc3RlbnRDb29raWUiOm51bGx9.VzhNaDhuMGEveG44NWNyT0hHUHdpT201N0RVeUdnM1IxUkFmc21KQVM4cz0": http2: client connection force closed via ClientConn.Close
2021/06/03 02:29:50 ERROR : Mimir Backups/Hyper-V VM Backups/2.5.2021/BlueIris.vhdx: Failed to copy: Put "https://hinsonventures-my.sharepoint.com/personal/jerry_reed_hinsoncompanies_com/_api/v2.0/drives/b!ksQgOXPXt0q5KsTGNXLgWS6pPDzqblhGoKy1o0_8xNX3OUCQWSQfRrKUHcpMU64d/items/016TXQIDTS2CRSFIQKBNCYPRZIIYO7UNVI/uploadSession?guid='8081c387-3e00-4092-84a0-6db85c151e09'&path='~tmp68_q31acqk1dnioos8hk7q9ua4ga4'&overwrite=True&rename=False&dc=0&tempauth=eyJ0eXAiOiJKV1QiLCJhbGciOiJub25lIn0.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDAvaGluc29udmVudHVyZXMtbXkuc2hhcmVwb2ludC5jb21ANWJmYTE0NTktM2VkMi00ZWRjLThhY2QtZjBjM2MyNDcwNjkzIiwiaXNzIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwIiwibmJmIjoiMTYyMjY3MTM0NCIsImV4cCI6IjE2MjI3NTc3NDQiLCJlbmRwb2ludHVybCI6InZacFByTTZBb0k2MTlzeGxZdlRST1c0cXZqaTd2NE5hazIyL2RHYmE2cW89IiwiZW5kcG9pbnR1cmxMZW5ndGgiOiIzMzciLCJpc2xvb3BiYWNrIjoiVHJ1ZSIsImNpZCI6IlkySXpaakE0T0RrdE5ETTRNaTAwTjJNMExXRXpPR010TldGbFpXUmpaalprTW1OayIsInZlciI6Imhhc2hlZHByb29mdG9rZW4iLCJzaXRlaWQiOiJNemt5TUdNME9USXRaRGMzTXkwMFlXSTNMV0k1TW1FdFl6UmpOak0xTnpKbE1EVTUiLCJhcHBfZGlzcGxheW5hbWUiOiJyY2xvbmUiLCJnaXZlbl9uYW1lIjoiSmVycnkiLCJmYW1pbHlfbmFtZSI6IlJlZWQiLCJhcHBpZCI6ImIxNTY2NWQ5LWVkYTYtNDA5Mi04NTM5LTBlZWMzNzZhZmQ1OSIsInRpZCI6IjViZmExNDU5LTNlZDItNGVkYy04YWNkLWYwYzNjMjQ3MDY5MyIsInVwbiI6ImplcnJ5LnJlZWRAaGluc29uY29tcGFuaWVzLmNvbSIsInB1aWQiOiIxMDAzMjAwMTQyRDRDOTZDIiwiY2FjaGVrZXkiOiIwaC5mfG1lbWJlcnNoaXB8MTAwMzIwMDE0MmQ0Yzk2Y0BsaXZlLmNvbSIsInNjcCI6Im15ZmlsZXMucmVhZCBhbGxmaWxlcy5yZWFkIG15ZmlsZXMud3JpdGUgYWxsZmlsZXMud3JpdGUgYWxsc2l0ZXMucmVhZCIsImFjcnMiOiJyMSxyMixyMyxjMSxjMixjMyIsInR0IjoiMiIsInVzZVBlcnNpc3RlbnRDb29raWUiOm51bGx9.bkl4L0VlZDQrTjlOLzNQQmNCOVdiYjY5citxZFZQYlNIT0Ztd0FScGpxbz0": http2: client connection force closed via ClientConn.Close
2021/06/03 02:29:50 ERROR : Mimir Backups/C9DF377EAD4805EA-00-00-JM-windows-6.9.2020.mrimg: Failed to copy: Put "https://hinsonventures-my.sharepoint.com/personal/jerry_reed_hinsoncompanies_com/_api/v2.0/drives/b!ksQgOXPXt0q5KsTGNXLgWS6pPDzqblhGoKy1o0_8xNX3OUCQWSQfRrKUHcpMU64d/items/016TXQIDT7SUSCXF2XIJGIWVJ2XQCIH4VS/uploadSession?guid='44ea1623-f97c-401f-b011-8ffa38eba7d6'&path='~tmp37_68qsaorpntiss2rfb1lr68n00ma2144dch2gso118lo4ldud196t94d4ngignn5qk49jf2h4v5gucumppcdb5v64r6e3q41dknbiep0'&overwrite=True&rename=False&dc=0&tempauth=eyJ0eXAiOiJKV1QiLCJhbGciOiJub25lIn0.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDAvaGluc29udmVudHVyZXMtbXkuc2hhcmVwb2ludC5jb21ANWJmYTE0NTktM2VkMi00ZWRjLThhY2QtZjBjM2MyNDcwNjkzIiwiaXNzIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwIiwibmJmIjoiMTYyMjY3MTM0MyIsImV4cCI6IjE2MjI3NTc3NDMiLCJlbmRwb2ludHVybCI6ImsrT0QzSTdPRFlSV1RlbWR6bnowRG45ZnFuTXhDU1NSbmpkb2pOTE5LdzQ9IiwiZW5kcG9pbnR1cmxMZW5ndGgiOiI0MTQiLCJpc2xvb3BiYWNrIjoiVHJ1ZSIsImNpZCI6IlpUWmlPVEV5TnpjdE9UaGxaQzAwTW1SaExUaGpaamd0TnpabVpHVm1OakZoT1RBMCIsInZlciI6Imhhc2hlZHByb29mdG9rZW4iLCJzaXRlaWQiOiJNemt5TUdNME9USXRaRGMzTXkwMFlXSTNMV0k1TW1FdFl6UmpOak0xTnpKbE1EVTUiLCJhcHBfZGlzcGxheW5hbWUiOiJyY2xvbmUiLCJnaXZlbl9uYW1lIjoiSmVycnkiLCJmYW1pbHlfbmFtZSI6IlJlZWQiLCJhcHBpZCI6ImIxNTY2NWQ5LWVkYTYtNDA5Mi04NTM5LTBlZWMzNzZhZmQ1OSIsInRpZCI6IjViZmExNDU5LTNlZDItNGVkYy04YWNkLWYwYzNjMjQ3MDY5MyIsInVwbiI6ImplcnJ5LnJlZWRAaGluc29uY29tcGFuaWVzLmNvbSIsInB1aWQiOiIxMDAzMjAwMTQyRDRDOTZDIiwiY2FjaGVrZXkiOiIwaC5mfG1lbWJlcnNoaXB8MTAwMzIwMDE0MmQ0Yzk2Y0BsaXZlLmNvbSIsInNjcCI6Im15ZmlsZXMucmVhZCBhbGxmaWxlcy5yZWFkIG15ZmlsZXMud3JpdGUgYWxsZmlsZXMud3JpdGUgYWxsc2l0ZXMucmVhZCIsImFjcnMiOiJyMSxyMixyMyxjMSxjMixjMyIsInR0IjoiMiIsInVzZVBlcnNpc3RlbnRDb29raWUiOm51bGx9.Q2tIZjExemlZNVhEUzd0NG5BZ0QwSllYVHlrMjlUSXJGdUlXQTAwR3ZYST0": http2: client connection force closed via ClientConn.Close
2021/06/03 02:42:19 ERROR : JMs Music and Photos/Photos Library/xad: Failed to copy: Put "https://hinsonventures-my.sharepoint.com/personal/jerry_reed_hinsoncompanies_com/_api/v2.0/drives/b!ksQgOXPXt0q5KsTGNXLgWS6pPDzqblhGoKy1o0_8xNX3OUCQWSQfRrKUHcpMU64d/items/016TXQIDW2HPI54PEI55D2GDRJOFUGSEPG/uploadSession?guid='4bd1405e-e56f-4574-830a-c5b055342362'&path='~tmp26_72ohuo4u71p8d069o98rsoh660'&overwrite=True&rename=False&dc=0&tempauth=eyJ0eXAiOiJKV1QiLCJhbGciOiJub25lIn0.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDAvaGluc29udmVudHVyZXMtbXkuc2hhcmVwb2ludC5jb21ANWJmYTE0NTktM2VkMi00ZWRjLThhY2QtZjBjM2MyNDcwNjkzIiwiaXNzIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwIiwibmJmIjoiMTYyMjcwMTc5MSIsImV4cCI6IjE2MjI3ODgxOTEiLCJlbmRwb2ludHVybCI6ImJDY0NzdjQrcytaRGtWYlRWTG9scFg3K25haEw5V041NjV5VlVONUF1OGs9IiwiZW5kcG9pbnR1cmxMZW5ndGgiOiIzMzciLCJpc2xvb3BiYWNrIjoiVHJ1ZSIsImNpZCI6IlpEWXdOR1JsWmpVdE9URTJPQzAwWldFekxXRTFNelV0WkdRNU5tUmlOR0ZrWm1RMiIsInZlciI6Imhhc2hlZHByb29mdG9rZW4iLCJzaXRlaWQiOiJNemt5TUdNME9USXRaRGMzTXkwMFlXSTNMV0k1TW1FdFl6UmpOak0xTnpKbE1EVTUiLCJhcHBfZGlzcGxheW5hbWUiOiJyY2xvbmUiLCJnaXZlbl9uYW1lIjoiSmVycnkiLCJmYW1pbHlfbmFtZSI6IlJlZWQiLCJhcHBpZCI6ImIxNTY2NWQ5LWVkYTYtNDA5Mi04NTM5LTBlZWMzNzZhZmQ1OSIsInRpZCI6IjViZmExNDU5LTNlZDItNGVkYy04YWNkLWYwYzNjMjQ3MDY5MyIsInVwbiI6ImplcnJ5LnJlZWRAaGluc29uY29tcGFuaWVzLmNvbSIsInB1aWQiOiIxMDAzMjAwMTQyRDRDOTZDIiwiY2FjaGVrZXkiOiIwaC5mfG1lbWJlcnNoaXB8MTAwMzIwMDE0MmQ0Yzk2Y0BsaXZlLmNvbSIsInNjcCI6Im15ZmlsZXMucmVhZCBhbGxmaWxlcy5yZWFkIG15ZmlsZXMud3JpdGUgYWxsZmlsZXMud3JpdGUgYWxsc2l0ZXMucmVhZCIsImFjcnMiOiJyMSxyMixyMyxjMSxjMixjMyIsInR0IjoiMiIsInVzZVBlcnNpc3RlbnRDb29raWUiOm51bGx9.eCtpSzVqc1VUamY4SHJWai84bUxMWXg5bE9Xa293V0QyYW4zODdkZEcxZz0": http2: client connection force closed via ClientConn.Close
2021/06/03 02:42:21 ERROR : JMs Music and Photos/Photos Library/xaf: Failed to copy: Put "https://hinsonventures-my.sharepoint.com/personal/jerry_reed_hinsoncompanies_com/_api/v2.0/drives/b!ksQgOXPXt0q5KsTGNXLgWS6pPDzqblhGoKy1o0_8xNX3OUCQWSQfRrKUHcpMU64d/items/016TXQIDW2HPI54PEI55D2GDRJOFUGSEPG/uploadSession?guid='4fb7a180-de26-4efc-888d-fc248e5b0773'&path='~tmpA2_u9jslpm2bp397gvsm9ki66je60'&overwrite=True&rename=False&dc=0&tempauth=eyJ0eXAiOiJKV1QiLCJhbGciOiJub25lIn0.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDAvaGluc29udmVudHVyZXMtbXkuc2hhcmVwb2ludC5jb21ANWJmYTE0NTktM2VkMi00ZWRjLThhY2QtZjBjM2MyNDcwNjkzIiwiaXNzIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwIiwibmJmIjoiMTYyMjcwMTc5MSIsImV4cCI6IjE2MjI3ODgxOTEiLCJlbmRwb2ludHVybCI6InF6R1g0QU1tcUtIc09RWkh5S2M0dzhUMVFza3dFNWEzUHJFQkF6Yk1tRmc9IiwiZW5kcG9pbnR1cmxMZW5ndGgiOiIzMzciLCJpc2xvb3BiYWNrIjoiVHJ1ZSIsImNpZCI6Ik9EaGpOekl5TVRNdE9EY3dPQzAwTkRrMExXSmtNemt0TVRBM05qQXpZemhrWTJReCIsInZlciI6Imhhc2hlZHByb29mdG9rZW4iLCJzaXRlaWQiOiJNemt5TUdNME9USXRaRGMzTXkwMFlXSTNMV0k1TW1FdFl6UmpOak0xTnpKbE1EVTUiLCJhcHBfZGlzcGxheW5hbWUiOiJyY2xvbmUiLCJnaXZlbl9uYW1lIjoiSmVycnkiLCJmYW1pbHlfbmFtZSI6IlJlZWQiLCJhcHBpZCI6ImIxNTY2NWQ5LWVkYTYtNDA5Mi04NTM5LTBlZWMzNzZhZmQ1OSIsInRpZCI6IjViZmExNDU5LTNlZDItNGVkYy04YWNkLWYwYzNjMjQ3MDY5MyIsInVwbiI6ImplcnJ5LnJlZWRAaGluc29uY29tcGFuaWVzLmNvbSIsInB1aWQiOiIxMDAzMjAwMTQyRDRDOTZDIiwiY2FjaGVrZXkiOiIwaC5mfG1lbWJlcnNoaXB8MTAwMzIwMDE0MmQ0Yzk2Y0BsaXZlLmNvbSIsInNjcCI6Im15ZmlsZXMucmVhZCBhbGxmaWxlcy5yZWFkIG15ZmlsZXMud3JpdGUgYWxsZmlsZXMud3JpdGUgYWxsc2l0ZXMucmVhZCIsImFjcnMiOiJyMSxyMixyMyxjMSxjMixjMyIsInR0IjoiMiIsInVzZVBlcnNpc3RlbnRDb29raWUiOm51bGx9.RUZwRDRRS2hTUkYvTWpZZWl5NitzbUpzK011RENDV2hlY0pUeTl2dVhOcz0": http2: client connection force closed via ClientConn.Close
2021/06/03 02:42:21 ERROR : Linode/NewProductionNode4122021.img: Failed to copy: Put "https://hinsonventures-my.sharepoint.com/personal/jerry_reed_hinsoncompanies_com/_api/v2.0/drives/b!ksQgOXPXt0q5KsTGNXLgWS6pPDzqblhGoKy1o0_8xNX3OUCQWSQfRrKUHcpMU64d/items/016TXQIDQJ4PMMHYTPUVBJAVCMIAVUZ2SX/uploadSession?guid='6a64c050-06d4-4386-9df6-f6a88656e66b'&path='~tmp46_s1l3cphjgiqbpqopavuesjjgrc0l0trck60ktgn582hk0boka4mg'&overwrite=True&rename=False&dc=0&tempauth=eyJ0eXAiOiJKV1QiLCJhbGciOiJub25lIn0.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDAvaGluc29udmVudHVyZXMtbXkuc2hhcmVwb2ludC5jb21ANWJmYTE0NTktM2VkMi00ZWRjLThhY2QtZjBjM2MyNDcwNjkzIiwiaXNzIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwIiwibmJmIjoiMTYyMjY3MTM0MyIsImV4cCI6IjE2MjI3NTc3NDMiLCJlbmRwb2ludHVybCI6IjEyRlVMMXVtNjRnS256UDFrZVJDVnV4RzQ1OXI0QkxjaDdQc3BmSTFkMHM9IiwiZW5kcG9pbnR1cmxMZW5ndGgiOiIzNjMiLCJpc2xvb3BiYWNrIjoiVHJ1ZSIsImNpZCI6Ik16a3daREl5TURNdE1UWXdOUzAwTURGbExXRTRZelF0TldRMlpUTTVZMlV4TnpZeSIsInZlciI6Imhhc2hlZHByb29mdG9rZW4iLCJzaXRlaWQiOiJNemt5TUdNME9USXRaRGMzTXkwMFlXSTNMV0k1TW1FdFl6UmpOak0xTnpKbE1EVTUiLCJhcHBfZGlzcGxheW5hbWUiOiJyY2xvbmUiLCJnaXZlbl9uYW1lIjoiSmVycnkiLCJmYW1pbHlfbmFtZSI6IlJlZWQiLCJhcHBpZCI6ImIxNTY2NWQ5LWVkYTYtNDA5Mi04NTM5LTBlZWMzNzZhZmQ1OSIsInRpZCI6IjViZmExNDU5LTNlZDItNGVkYy04YWNkLWYwYzNjMjQ3MDY5MyIsInVwbiI6ImplcnJ5LnJlZWRAaGluc29uY29tcGFuaWVzLmNvbSIsInB1aWQiOiIxMDAzMjAwMTQyRDRDOTZDIiwiY2FjaGVrZXkiOiIwaC5mfG1lbWJlcnNoaXB8MTAwMzIwMDE0MmQ0Yzk2Y0BsaXZlLmNvbSIsInNjcCI6Im15ZmlsZXMucmVhZCBhbGxmaWxlcy5yZWFkIG15ZmlsZXMud3JpdGUgYWxsZmlsZXMud3JpdGUgYWxsc2l0ZXMucmVhZCIsImFjcnMiOiJyMSxyMixyMyxjMSxjMixjMyIsInR0IjoiMiIsInVzZVBlcnNpc3RlbnRDb29raWUiOm51bGx9.ZmZZdUI3UUxwWlpKb1dOa2xlZE1kRmpTd0cwRk1ZUTYvWlVhaHE5T3lTUT0": http2: client connection force closed via ClientConn.Close
2021/06/03 03:58:45 INFO  : JMs Music and Photos/Photos Library/xac: Copied (new)
2021/06/03 03:58:45 ERROR : Encrypted drive 'cryptonedrive:Backups/Bulk-Data/Vault/Backups': not deleting files as there were IO errors
2021/06/03 03:58:45 ERROR : Encrypted drive 'cryptonedrive:Backups/Bulk-Data/Vault/Backups': not deleting directories as there were IO errors
2021/06/03 03:58:45 ERROR : Attempt 1/3 failed with 11 errors and: Put "https://hinsonventures-my.sharepoint.com/personal/jerry_reed_hinsoncompanies_com/_api/v2.0/drives/b!ksQgOXPXt0q5KsTGNXLgWS6pPDzqblhGoKy1o0_8xNX3OUCQWSQfRrKUHcpMU64d/items/016TXQIDQJ4PMMHYTPUVBJAVCMIAVUZ2SX/uploadSession?guid='6a64c050-06d4-4386-9df6-f6a88656e66b'&path='~tmp46_s1l3cphjgiqbpqopavuesjjgrc0l0trck60ktgn582hk0boka4mg'&overwrite=True&rename=False&dc=0&tempauth=eyJ0eXAiOiJKV1QiLCJhbGciOiJub25lIn0.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDAvaGluc29udmVudHVyZXMtbXkuc2hhcmVwb2ludC5jb21ANWJmYTE0NTktM2VkMi00ZWRjLThhY2QtZjBjM2MyNDcwNjkzIiwiaXNzIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwIiwibmJmIjoiMTYyMjY3MTM0MyIsImV4cCI6IjE2MjI3NTc3NDMiLCJlbmRwb2ludHVybCI6IjEyRlVMMXVtNjRnS256UDFrZVJDVnV4RzQ1OXI0QkxjaDdQc3BmSTFkMHM9IiwiZW5kcG9pbnR1cmxMZW5ndGgiOiIzNjMiLCJpc2xvb3BiYWNrIjoiVHJ1ZSIsImNpZCI6Ik16a3daREl5TURNdE1UWXdOUzAwTURGbExXRTRZelF0TldRMlpUTTVZMlV4TnpZeSIsInZlciI6Imhhc2hlZHByb29mdG9rZW4iLCJzaXRlaWQiOiJNemt5TUdNME9USXRaRGMzTXkwMFlXSTNMV0k1TW1FdFl6UmpOak0xTnpKbE1EVTUiLCJhcHBfZGlzcGxheW5hbWUiOiJyY2xvbmUiLCJnaXZlbl9uYW1lIjoiSmVycnkiLCJmYW1pbHlfbmFtZSI6IlJlZWQiLCJhcHBpZCI6ImIxNTY2NWQ5LWVkYTYtNDA5Mi04NTM5LTBlZWMzNzZhZmQ1OSIsInRpZCI6IjViZmExNDU5LTNlZDItNGVkYy04YWNkLWYwYzNjMjQ3MDY5MyIsInVwbiI6ImplcnJ5LnJlZWRAaGluc29uY29tcGFuaWVzLmNvbSIsInB1aWQiOiIxMDAzMjAwMTQyRDRDOTZDIiwiY2FjaGVrZXkiOiIwaC5mfG1lbWJlcnNoaXB8MTAwMzIwMDE0MmQ0Yzk2Y0BsaXZlLmNvbSIsInNjcCI6Im15ZmlsZXMucmVhZCBhbGxmaWxlcy5yZWFkIG15ZmlsZXMud3JpdGUgYWxsZmlsZXMud3JpdGUgYWxsc2l0ZXMucmVhZCIsImFjcnMiOiJyMSxyMixyMyxjMSxjMixjMyIsInR0IjoiMiIsInVzZVBlcnNpc3RlbnRDb29raWUiOm51bGx9.ZmZZdUI3UUxwWlpKb1dOa2xlZE1kRmpTd0cwRk1ZUTYvWlVhaHE5T3lTUT0": http2: client connection force closed via ClientConn.Close
2021/06/03 12:05:52 INFO  : JMs Music and Photos/Photos Library/xad: Copied (new)
2021/06/03 12:06:51 INFO  : Danielles Music and Photos/xaa: Copied (new)
2021/06/03 12:07:45 ERROR : JMs Music and Photos/Photos Library/xaf: Failed to copy: corrupted on transfer: QuickXorHash crypted hash differ "edf4495d493fd6314762cf7f6fc96322ecda6428" vs "164e9665d2e08da9604344db8a435bca9997800c"
2021/06/03 14:47:08 INFO  : Signal received: interrupt

hello,
not sure the exact problem.
there is a typo in the command
--mas-transfer should be --max-transfer

would be helpful to see the exact command and the top of the debug log.

Thanks for catching that. I had to manually retype the command into this forum so that was my bad but its correct in the script that runs it. I am going to correct the typo above.

if you post the top of the rclone debug log, which contains the command, no need to manually type.

are any of the source files changing during the time rclone is running?

have you done this?
https://rclone.org/onedrive/#getting-your-own-client-id-and-key

No, I am not manipulating any of the files while its uploading. I did just upload these same files to Google Drive and had no issues uploading them FWIW.

EDIT: Yes I did it and have implemented in the config file.

I suggest you try to isolate/simplify the issue, that will make it much easier to help you. Right now, the issue can be related to many different parameters: rclone parameter settings, rclone encryption, rclone communication with OneDrive, the settings of your business OneDrive, your network connection, …

You already successfully tried unencrypted upload using OneDrive in a web browser, and that is a good starting point.

Now consider other ideas to further reduce/isolate the issue. Can the issue be reproduced:

  • With less parameters (that is without --bwlimit=off --max-transfer=off --cutoff-mode=cautious --transfers 8 --onedrive-no-versions)
  • Without using rclone mount? (if used)
  • Without encryption?
  • With a single (large) file (or folder)?
  • With a personal OneDrive?
  • On another computer?
  • On another network connection?

These were my first thoughts as inspiration, other ideas may be better/more relevant in your situation.

1 Like

Thanks for the reply. I am in the process of testing some of these and will report back. This will take some time as each sync command takes several hours to run long enough to get meaningful data from.

My first test is to run the command with ONLY the -vv and --stats 1m flags. all others have been removed.

will report back.

Try adding the following flag: --user-agent "ISV|rclone.org|rclone/v1.55.1"

Seems sometimes OneDrive doesn't like if the user agent doesn't match and has fixed a lot of issues for me.

Just started a new sync with that added. Ill post with the results when its finished.

I also just finished running a sync (not using the above add on) and I am attaching the log.

Log: https://hinsonventures-my.sharepoint.com/:u:/g/personal/jmhinson_hinsoncompanies_com/EXrNtcO67ZtEm_yBU2CPY1oBEg-glGTSOyhOlBybsuaAjw?e=Jmhxhv

The command failed and I got this response in the log after adding that. Let me know if I made a typo when entering that option.

2021/06/08 09:13:32 DEBUG : Using config file from "/home/jm/.config/rclone/rclone.conf"

2021/06/08 09:13:32 DEBUG : rclone: Version "v1.55.1" starting with parameters ["/usr/bin/rclone" "sync" "-vv" "/run/media/jm/BackupMedia/Backups/GoogleDriveBackup/Vault/Backups" "cryptonedrive:Backups/Bulk-Data/Vault/Backups" "--log-file=/tmp/rclone-uploader/rclone-temp2.log" "--stats" "1m" "--user-agent" "ISV|rclone.org|rclone/v1.55.1"]

2021/06/08 09:13:32 DEBUG : Creating backend with remote "/run/media/jm/BackupMedia/Backups/GoogleDriveBackup/Vault/Backups"

2021/06/08 09:13:32 DEBUG : Creating backend with remote "cryptonedrive:Backups/Bulk-Data/Vault/Backups"

2021/06/08 09:13:32 DEBUG : Creating backend with remote "onedrive:e9d1diskbigi3us0p13gitugn0/jaiclgqaiv6n4rt3u5emi6hokk/uehs32gavujifuca79p9niudog/e9d1diskbigi3us0p13gitugn0"

2021/06/08 09:13:33 DEBUG : pacer: low level retry 1/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:33 DEBUG : pacer: Rate limited, increasing sleep to 20ms

2021/06/08 09:13:33 DEBUG : pacer: low level retry 2/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:33 DEBUG : pacer: Rate limited, increasing sleep to 40ms

2021/06/08 09:13:34 DEBUG : pacer: Reducing sleep to 30ms

2021/06/08 09:13:34 DEBUG : pacer: low level retry 1/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:34 DEBUG : pacer: Rate limited, increasing sleep to 60ms

2021/06/08 09:13:35 DEBUG : pacer: low level retry 2/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:35 DEBUG : pacer: Rate limited, increasing sleep to 120ms

2021/06/08 09:13:35 DEBUG : pacer: low level retry 3/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:35 DEBUG : pacer: Rate limited, increasing sleep to 240ms

2021/06/08 09:13:36 DEBUG : pacer: low level retry 4/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:36 DEBUG : pacer: Rate limited, increasing sleep to 480ms

2021/06/08 09:13:37 DEBUG : pacer: low level retry 5/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:37 DEBUG : pacer: Rate limited, increasing sleep to 960ms

2021/06/08 09:13:38 DEBUG : pacer: low level retry 6/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:38 DEBUG : pacer: Rate limited, increasing sleep to 1.92s

2021/06/08 09:13:39 DEBUG : pacer: low level retry 7/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:39 DEBUG : pacer: Rate limited, increasing sleep to 2s

2021/06/08 09:13:40 DEBUG : pacer: low level retry 8/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:42 DEBUG : pacer: low level retry 9/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:44 DEBUG : pacer: low level retry 10/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:46 DEBUG : pacer: low level retry 1/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:48 DEBUG : pacer: low level retry 2/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:50 DEBUG : pacer: low level retry 3/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:52 DEBUG : pacer: low level retry 4/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:54 DEBUG : pacer: Reducing sleep to 1.5s

2021/06/08 09:13:56 DEBUG : pacer: low level retry 1/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:13:56 DEBUG : pacer: Rate limited, increasing sleep to 2s

2021/06/08 09:13:58 DEBUG : pacer: Reducing sleep to 1.5s

2021/06/08 09:14:00 DEBUG : pacer: low level retry 1/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:00 DEBUG : pacer: Rate limited, increasing sleep to 2s

2021/06/08 09:14:02 DEBUG : pacer: low level retry 2/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:03 DEBUG : pacer: low level retry 3/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:06 DEBUG : pacer: low level retry 4/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:07 DEBUG : pacer: low level retry 5/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:09 DEBUG : pacer: low level retry 6/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:12 DEBUG : pacer: Reducing sleep to 1.5s

2021/06/08 09:14:13 DEBUG : pacer: low level retry 1/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:13 DEBUG : pacer: Rate limited, increasing sleep to 2s

2021/06/08 09:14:15 DEBUG : pacer: low level retry 2/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:17 DEBUG : pacer: low level retry 3/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:19 DEBUG : pacer: Reducing sleep to 1.5s

2021/06/08 09:14:21 DEBUG : pacer: low level retry 1/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:21 DEBUG : pacer: Rate limited, increasing sleep to 2s

2021/06/08 09:14:22 DEBUG : pacer: Reducing sleep to 1.5s

2021/06/08 09:14:24 DEBUG : pacer: low level retry 1/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:24 DEBUG : pacer: Rate limited, increasing sleep to 2s

2021/06/08 09:14:26 DEBUG : pacer: Reducing sleep to 1.5s

2021/06/08 09:14:26 DEBUG : Creating backend with remote "onedrive:e9d1diskbigi3us0p13gitugn0/jaiclgqaiv6n4rt3u5emi6hokk/uehs32gavujifuca79p9niudog/e9d1diskbigi3us0p13gitugn0"

2021/06/08 09:14:26 DEBUG : pacer: low level retry 1/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:26 DEBUG : pacer: Rate limited, increasing sleep to 20ms

2021/06/08 09:14:26 DEBUG : pacer: low level retry 2/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:26 DEBUG : pacer: Rate limited, increasing sleep to 40ms

2021/06/08 09:14:27 DEBUG : pacer: low level retry 3/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:27 DEBUG : pacer: Rate limited, increasing sleep to 80ms

2021/06/08 09:14:27 DEBUG : pacer: low level retry 4/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:27 DEBUG : pacer: Rate limited, increasing sleep to 160ms

2021/06/08 09:14:28 DEBUG : pacer: low level retry 5/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:28 DEBUG : pacer: Rate limited, increasing sleep to 320ms

2021/06/08 09:14:28 DEBUG : pacer: low level retry 6/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:28 DEBUG : pacer: Rate limited, increasing sleep to 640ms

2021/06/08 09:14:28 DEBUG : pacer: low level retry 7/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:28 DEBUG : pacer: Rate limited, increasing sleep to 1.28s

2021/06/08 09:14:29 DEBUG : pacer: low level retry 8/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:29 DEBUG : pacer: Rate limited, increasing sleep to 2s

2021/06/08 09:14:30 DEBUG : pacer: low level retry 9/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:32 DEBUG : pacer: low level retry 10/10 (error serviceNotAvailable: Service unavailable)

2021/06/08 09:14:32 Failed to create file system for "cryptonedrive:Backups/Bulk-Data/Vault/Backups": 

Looks like you are already limited by OneDrive. Let it sit for a couple of hours and retry again with the flag. You are getting service not available error.

Hi jcastill, I am curious, maybe your tip should go into the documentation or rclone. Can you point me to the source of your proposed user agent? (Preferably a standard, Microsoft document or similar)

Sorry for the delay. Had to search again for the source. Check here: Avoid getting throttled or blocked in SharePoint Online | Microsoft Docs

If you scroll down to What are the recommendations: Avoid getting throttled or blocked in SharePoint Online | Microsoft Docs

You will find Microsoft recommends the user-agent: ISV|CompanyName|AppName/Version

This also shows multiple reasons on being rate limited and how to avoid it.

Now, answering your question. I think it would be best to open a ticket and have Rclone use the specific format asked by OneDrive as user-agent (unless we override) and add to Documentation this source and tips if somebody wants to use another user-agent.

Hope this helps!

1 Like

No problem, excellent tip and reference!

I have created the issue: Add tip to OneDrive documentation to avoid SharePoint throttling/blocking. I encourage you to add your preferred wording or do the entire PR if you fancy.

@TheJM Sorry for the detour, how are you doing?

1 Like

I ran the command with and without the below add on

--user-agent "ISV|rclone.org|rclone/v1.55.1"

and am stiff getting a lot of service not available errors (as shown above) as well as QuickXorHash not matching. so unfortunately doesn't seem like this add-on did anything at least on the 2 syncs i ran after adding it.

I have also eliminated most all of the parameters that i was using but that didnt help either. I have posted 1 long log and 1 shorter one above for reference.

I would look to y'all for ideas.

For my case, any time I tried more than --transfers=4 resulted in throttling.

1 Like

Thanks. I will try that out.

Also try --transfers=2 like hsai recommended. You can also add --tpslimit 2 to try and slow it down.

If you are still getting errors. Please post new log. Also if you get an error, stop rclone and do not try to restart it in an hour to make sure the throttling is over. Otherwise you might end up getting banned.

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.