Upload large files to OneDrive for Business

Help and Support
1

What is the problem you are having with rclone?

the --multi-thread-streams=N option doesn't seem to work when copying files from local storage to OneDrive remote

Run the command 'rclone version' and share the full output of the command.

- os/version: Microsoft Windows 10 Pro 1903 (64 bit)
- os/kernel: 10.0.18362.239 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.17.2
- go/linking: dynamic
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

OneDrive for Business

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy test.iso Onedrive: --multi-thread-streams=8

The rclone config contents with secrets removed.

[Onedrive]
type = onedrive
token = {"access_token":"REMOVED","expiry":"2022-03-20T00:43:15.1112961+03:00"}
drive_id = REMOVED
drive_type = business

A log from the command with the -vv flag

rclone copy test.iso Onedrive: --multi-thread-streams=8 -vv
2022/03/28 22:52:52 DEBUG : rclone: Version "v1.57.0" starting with parameters ["rclone" "copy" "test.iso" "Onedrive:" "--multi-thread-streams=8" "-vv"]
2022/03/28 22:52:52 DEBUG : Creating backend with remote "test.iso"
2022/03/28 22:52:52 DEBUG : Using config file from "C:\\Program Files (x86)\\rclone-v1.57.0-windows-amd64\\rclone.conf"
2022/03/28 22:52:52 DEBUG : fs cache: adding new entry for parent of "test.iso", "//?/D:/Downloads/Ansys.Products.2021.R2.PDF.Docs"
2022/03/28 22:52:52 DEBUG : Creating backend with remote "Onedrive:"
2022/03/28 22:52:52 DEBUG : One drive root '': Token expired but no uploads in progress - doing nothing
2022/03/28 22:52:52 DEBUG : Onedrive: Loaded invalid token from config file - ignoring
2022/03/28 22:52:54 DEBUG : Saving config "token" in section "Onedrive" of the config file
2022/03/28 22:53:00 DEBUG : Onedrive: Saved new token in config file
2022/03/28 22:53:01 DEBUG : test.iso: Need to transfer - File not found at Destination
2022/03/28 22:53:01 DEBUG : test.iso: Starting multipart upload
2022/03/28 22:53:02 DEBUG : test.iso: Uploading segment 0/2045569024 size 10485760
2022/03/28 22:53:06 INFO  : Signal received: interrupt
2022/03/28 22:53:06 DEBUG : test.iso: Cancelling multipart upload: <nil>
2022/03/28 22:53:06 INFO  : Exiting...
2

hi,
--multi-thread-stream is for downloads, not uploads
"When using multi thread downloads"

for uploads,
--- create a client id/secret
--- tweak chunk size

3

I have created a client id/secret but nothing much changed

also, how would changing the chunk size affect the upload speed? rclone will upload using a single thread and thus the speed won't be affected.

finally, the Onedrive API appears to support multi-threaded uploaded as it can be seen here

4

it is recommended to do that. tho might not make a difference in all cases.

less api calls, can reduce throttling.

did a quick read, did not see anything about multi-threaded uploads.
in fact, just the opposite behavior, single thread, chunk at a time, same as rclone
"The fragments of the file must be uploaded sequentially in order. Uploading fragments out of order will result in an error."

notice that
"If your app splits a file into multiple byte ranges, the size of each byte range MUST be a multiple of 320 KiB (327,680 bytes)"
which is the exact same as rclone
"Chunk size to upload files with - must be multiple of 320k (327,680 bytes)."

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.