OneDrive Performance testing

SunnyLo · April 30, 2024, 3:49pm

I know my Internet connection has a one gigabit uplink, thus it should be able to transfer in around 125MB/s, however when I try to download a file from OneDrive, I only get around 5-10 MB/s, which is not normal

sunnylo@SunnyLos-Laptop Downloads % ~/Documents/bin/rclone copy -P -vvv "od-seed:/Files/mkv.mkv" . 
2024/04/30 23:31:57 DEBUG : rclone: Version "v1.66.0" starting with parameters ["/Users/sunnylo/Documents/bin/rclone" "copy" "-P" "-vvv" "od-seed:/Files/mkv.mkv" "."]
2024/04/30 23:31:57 DEBUG : Creating backend with remote "od-seed:/Files/mkv.mkv"
2024/04/30 23:31:57 DEBUG : Using config file from "/Users/sunnylo/.config/rclone/rclone.conf"
2024/04/30 23:31:57 DEBUG : OneDrive root 'Files/mkv.mkv': Token expired but no uploads in progress - doing nothing
2024/04/30 23:31:57 DEBUG : od-seed: Loaded invalid token from config file - ignoring
2024/04/30 23:31:57 DEBUG : Saving config "token" in section "od-seed" of the config file
2024/04/30 23:31:57 DEBUG : od-seed: Saved new token in config file
2024/04/30 23:31:59 DEBUG : fs cache: adding new entry for parent of "od-seed:/Files/mkv.mkv", "od-seed:Files"
2024/04/30 23:31:59 DEBUG : Creating backend with remote "."
2024/04/30 23:31:59 DEBUG : fs cache: renaming cache item "." to be canonical "/Users/sunnylo/Downloads"
2024/04/30 23:31:59 DEBUG : mkv.mkv: Need to transfer - File not found at Destination
2024/04/30 23:31:59 DEBUG : mkv.mkv: multi-thread copy: disabling buffering because destination uses OpenWriterAt
2024/04/30 23:31:59 DEBUG : mkv.mkv: multi-thread copy: write buffer set to 131072
2024/04/30 23:31:59 DEBUG : mkv.mkv: multi-thread copy: using backend concurrency of 4 instead of --multi-thread-streams 4
2024/04/30 23:31:59 DEBUG : mkv.mkv: Starting multi-thread copy with 379 chunks of size 64Mi with 4 parallel streams
2024/04/30 23:31:59 DEBUG : mkv.mkv: multi-thread copy: chunk 4/379 (201326592-268435456) size 64Mi starting
2024/04/30 23:31:59 DEBUG : mkv.mkv: multi-thread copy: chunk 3/379 (134217728-201326592) size 64Mi starting
2024/04/30 23:31:59 DEBUG : mkv.mkv: multi-thread copy: chunk 1/379 (0-67108864) size 64Mi starting
2024/04/30 23:31:59 DEBUG : mkv.mkv: multi-thread copy: chunk 2/379 (67108864-134217728) size 64Mi starting
2024/04/30 23:32:00 DEBUG : mkv.mkv.pahelob4.partial: writing chunk 2
2024/04/30 23:32:00 DEBUG : mkv.mkv.pahelob4.partial: writing chunk 0
2024/04/30 23:32:00 DEBUG : mkv.mkv.pahelob4.partial: writing chunk 1
2024/04/30 23:32:00 DEBUG : mkv.mkv.pahelob4.partial: writing chunk 3
2024/04/30 23:32:09 DEBUG : mkv.mkv: multi-thread copy: chunk 4/379 (201326592-268435456) size 64Mi finished
2024/04/30 23:32:09 DEBUG : mkv.mkv: multi-thread copy: chunk 5/379 (268435456-335544320) size 64Mi starting
2024/04/30 23:32:11 DEBUG : mkv.mkv: multi-thread copy: chunk 3/379 (134217728-201326592) size 64Mi finished
2024/04/30 23:32:11 DEBUG : mkv.mkv: multi-thread copy: chunk 6/379 (335544320-402653184) size 64Mi starting
2024/04/30 23:32:27 DEBUG : mkv.mkv: multi-thread copy: chunk 1/379 (0-67108864) size 64Mi finished
2024/04/30 23:32:27 DEBUG : mkv.mkv: multi-thread copy: chunk 7/379 (402653184-469762048) size 64Mi starting
2024/04/30 23:32:27 DEBUG : mkv.mkv.pahelob4.partial: writing chunk 4
2024/04/30 23:32:30 DEBUG : mkv.mkv: multi-thread copy: chunk 2/379 (67108864-134217728) size 64Mi finished
2024/04/30 23:32:30 DEBUG : mkv.mkv: multi-thread copy: chunk 8/379 (469762048-536870912) size 64Mi starting
2024/04/30 23:32:30 DEBUG : mkv.mkv.pahelob4.partial: writing chunk 5
2024/04/30 23:32:33 DEBUG : mkv.mkv.pahelob4.partial: writing chunk 6
2024/04/30 23:32:37 INFO  : Signal received: interrupt
2024/04/30 23:32:37 INFO  : mkv.mkv.pahelob4.partial: Removing failed copy
2024/04/30 23:32:37 DEBUG : mkv.mkv: multi-thread copy: cancelling transfer on exit
2024/04/30 23:32:37 DEBUG : mkv.mkv: multi-thread copy: chunk 8/379 failed: multi-thread copy: failed to open source: Get "https://redacted": context canceled
2024/04/30 23:32:37 DEBUG : mkv.mkv: Reopening on read failure after offset 38970 bytes: retry 1/10: context canceled
2024/04/30 23:32:37 DEBUG : mkv.mkv: Reopening on read failure after offset 34549914 bytes: retry 1/10: context canceled
2024/04/30 23:32:37 DEBUG : mkv.mkv: Reopening on read failure after offset 54612246 bytes: retry 1/10: context canceled
2024/04/30 23:32:37 DEBUG : mkv.mkv: multi-thread copy: chunk 9/379 (536870912-603979776) size 64Mi starting
2024/04/30 23:32:37 DEBUG : mkv.mkv: Reopen failed after offset 38970 bytes read: Get "https://redacted": context canceled
2024/04/30 23:32:37 DEBUG : mkv.mkv: multi-thread copy: chunk 7/379 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:32:37 DEBUG : mkv.mkv: Reopen failed after offset 34549914 bytes read: Get "https://redacted": context canceled
2024/04/30 23:32:37 DEBUG : mkv.mkv: multi-thread copy: chunk 6/379 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:32:37 DEBUG : mkv.mkv: Reopen failed after offset 54612246 bytes read: Get "https://redacted": context canceled
2024/04/30 23:32:37 DEBUG : mkv.mkv: multi-thread copy: chunk 5/379 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:32:37 DEBUG : mkv.mkv: multi-thread copy: chunk 9/379 failed: multi-thread copy: failed to open source: Get "https://redacted": context canceled
2024/04/30 23:32:37 DEBUG : mkv.mkv: multi-thread copy: abort failed: multi-thread copy: failed to find temp file when aborting chunk writer: object not found
2024/04/30 23:32:37 INFO  : Exiting...
Transferred:   	  340.875 MiB / 23.628 GiB, 1%, 8.996 MiB/s, ETA 44m11s
Transferred:            0 / 1, 0%
Elapsed time:        40.9s
Transferring:
 * mkv.mkv:  1% /23.628Gi, 8.996Mi/s, 44m11s%

To verify my internet speed, I have run iperf3 with a close server

sunnylo@SunnyLos-Laptop Downloads % iperf3-darwin -c speedtest.hkg12.hk.leaseweb.net -p 5201-5210 
Connecting to host speedtest.hkg12.hk.leaseweb.net, port 5201
[  7] local 192.168.0.10 port 56689 connected to 64.120.117.34 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd          RTT
[  7]   0.00-1.00   sec   101 MBytes   845 Mbits/sec    0   2.78 MBytes   25ms     
[  7]   1.00-2.00   sec   104 MBytes   872 Mbits/sec    0   2.82 MBytes   26ms     
[  7]   2.00-3.00   sec   106 MBytes   886 Mbits/sec    0   2.80 MBytes   26ms     
[  7]   3.00-4.00   sec   106 MBytes   886 Mbits/sec    0   2.70 MBytes   23ms     
[  7]   4.00-5.00   sec   105 MBytes   877 Mbits/sec    0   2.80 MBytes   28ms     
[  7]   5.00-6.00   sec   105 MBytes   885 Mbits/sec    0   2.77 MBytes   26ms     
[  7]   6.00-7.00   sec   104 MBytes   869 Mbits/sec    0   2.49 MBytes   19ms     
[  7]   7.00-8.00   sec   104 MBytes   874 Mbits/sec    0   2.77 MBytes   26ms     
[  7]   8.00-9.00   sec   106 MBytes   887 Mbits/sec    0   2.74 MBytes   25ms     
[  7]   9.00-10.00  sec  99.9 MBytes   838 Mbits/sec    0   1.97 MBytes   28ms     
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  7]   0.00-10.00  sec  1.01 GBytes   872 Mbits/sec    0             sender
[  7]   0.00-10.01  sec  1.01 GBytes   869 Mbits/sec                  receiver

iperf Done.

So it shouldn't be a problem with my internet connection. Will it be a problem with OneDrive? Maybe the server doesn't have such bandwidth? To test this theory, I have try to download a OneDrive file through aria2, a parallel-downloader

sunnylo@SunnyLos-Laptop Downloads % aria2c ""https://redacted"
04/30 23:41:36 [NOTICE] Downloading 1 item(s)

04/30 23:41:36 [NOTICE] CUID#7 - Redirecting to "https://redacted"[#95359e 5.7MiB/37GiB(0%) CN:16 DL:21MiB ETA:30m10s]                                                                                                    
04/30 23:41:37 [NOTICE] CUID#9 - Redirecting to "https://redacted"
04/30 23:41:37 [NOTICE] CUID#16 - Redirecting to "https://redacted"
04/30 23:41:37 [NOTICE] CUID#17 - Redirecting to "https://redacted"
04/30 23:41:37 [NOTICE] CUID#15 - Redirecting to "https://redacted"
04/30 23:41:37 [NOTICE] CUID#10 - Redirecting to "https://redacted"
04/30 23:41:37 [NOTICE] CUID#22 - Redirecting to "https://redacted"
04/30 23:41:37 [NOTICE] CUID#11 - Redirecting to "https://redacted"
04/30 23:41:37 [NOTICE] CUID#18 - Redirecting to "https://redacted"
04/30 23:41:37 [NOTICE] CUID#20 - Redirecting to "https://redacted"
04/30 23:41:37 [NOTICE] CUID#21 - Redirecting to "https://redacted"
04/30 23:41:37 [NOTICE] CUID#8 - Redirecting to "https://redacted"
04/30 23:41:37 [NOTICE] CUID#13 - Redirecting to "https://redacted"
04/30 23:41:37 [NOTICE] CUID#19 - Redirecting to "https://redacted"
04/30 23:41:37 [NOTICE] CUID#12 - Redirecting to "https://redacted"
04/30 23:41:37 [NOTICE] CUID#14 - Redirecting to "https://redacted"
04/30 23:41:37 [NOTICE] CUID#20 - Redirecting to "https://redacted"
04/30 23:41:37 [NOTICE] CUID#18 - Redirecting to "https://redacted"[#95359e 574MiB/37GiB(1%) CN:16 DL:47MiB ETA:13m22s]^C                                                                                                  
04/30 23:41:49 [NOTICE] Shutdown sequence commencing... Press Ctrl-C again for emergency shutdown.

04/30 23:41:49 [NOTICE] Download GID#95359e58795dc498 not complete: /Users/sunnylo/Downloads/mkv.mkv

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
95359e|INPR|    47MiB/s|/Users/sunnylo/Downloads/mkv.mkv

Status Legend:
(INPR):download in-progress.

aria2 will resume download if the transfer is restarted.
If there are any errors, then see the log file. See '-l' option in help/man page for details.

I got 47MiB/s during the download, what? Seems to be some issue with rclone? The OneDrive server seems fine, maybe change to parallel download? Multi thread? Larger chunk size?

sunnylo@SunnyLos-Laptop Downloads % rclone copy -P -vv --multi-thread-chunk-size 2G --multi-thread-streams 16 "od-seed:/Files/mkv.mkv" .
2024/04/30 23:45:24 DEBUG : rclone: Version "v1.66.0" starting with parameters ["/Users/sunnylo/Documents/bin/rclone" "--multi-thread-chunk-size" "2G" "--multi-thread-streams" "16" "copy" "-P" "-vv" "--multi-thread-chunk-size" "2G" "--multi-thread-streams" "16" "od-seed:/Files/mkv.mkv" "."]
2024/04/30 23:45:24 DEBUG : Creating backend with remote "od-seed:/Files/mkv.mkv"
2024/04/30 23:45:24 DEBUG : Using config file from "/Users/sunnylo/.config/rclone/rclone.conf"
2024/04/30 23:45:25 DEBUG : fs cache: adding new entry for parent of "od-seed:/Files/mkv.mkv", "od-seed:Files"
2024/04/30 23:45:25 DEBUG : Creating backend with remote "."
2024/04/30 23:45:25 DEBUG : fs cache: renaming cache item "." to be canonical "/Users/sunnylo/Downloads"
2024/04/30 23:45:26 DEBUG : mkv.mkv: Need to transfer - File not found at Destination
2024/04/30 23:45:26 DEBUG : mkv.mkv: multi-thread copy: disabling buffering because destination uses OpenWriterAt
2024/04/30 23:45:26 DEBUG : mkv.mkv: multi-thread copy: write buffer set to 131072
2024/04/30 23:45:26 DEBUG : mkv.mkv: multi-thread copy: number of streams 16 was bigger than number of chunks 12
2024/04/30 23:45:26 DEBUG : mkv.mkv: Starting multi-thread copy with 12 chunks of size 2Gi with 12 parallel streams
2024/04/30 23:45:26 DEBUG : mkv.mkv: multi-thread copy: chunk 2/12 (2147483648-4294967296) size 2Gi starting
2024/04/30 23:45:26 DEBUG : mkv.mkv: multi-thread copy: chunk 11/12 (21474836480-23622320128) size 2Gi starting
2024/04/30 23:45:26 DEBUG : mkv.mkv: multi-thread copy: chunk 12/12 (23622320128-25370737554) size 1.628Gi starting
2024/04/30 23:45:26 DEBUG : mkv.mkv: multi-thread copy: chunk 10/12 (19327352832-21474836480) size 2Gi starting
2024/04/30 23:45:26 DEBUG : mkv.mkv: multi-thread copy: chunk 7/12 (12884901888-15032385536) size 2Gi starting
2024/04/30 23:45:26 DEBUG : mkv.mkv: multi-thread copy: chunk 5/12 (8589934592-10737418240) size 2Gi starting
2024/04/30 23:45:26 DEBUG : mkv.mkv: multi-thread copy: chunk 6/12 (10737418240-12884901888) size 2Gi starting
2024/04/30 23:45:26 DEBUG : mkv.mkv: multi-thread copy: chunk 1/12 (0-2147483648) size 2Gi starting
2024/04/30 23:45:26 DEBUG : mkv.mkv: multi-thread copy: chunk 9/12 (17179869184-19327352832) size 2Gi starting
2024/04/30 23:45:26 DEBUG : mkv.mkv: multi-thread copy: chunk 8/12 (15032385536-17179869184) size 2Gi starting
2024/04/30 23:45:26 DEBUG : mkv.mkv: multi-thread copy: chunk 4/12 (6442450944-8589934592) size 2Gi starting
2024/04/30 23:45:26 DEBUG : mkv.mkv: multi-thread copy: chunk 3/12 (4294967296-6442450944) size 2Gi starting
2024/04/30 23:45:27 DEBUG : mkv.mkv.caxuzik9.partial: writing chunk 10
2024/04/30 23:45:27 DEBUG : mkv.mkv.caxuzik9.partial: writing chunk 1
2024/04/30 23:45:28 DEBUG : mkv.mkv.caxuzik9.partial: writing chunk 2
2024/04/30 23:45:28 DEBUG : mkv.mkv.caxuzik9.partial: writing chunk 6
2024/04/30 23:45:28 DEBUG : mkv.mkv.caxuzik9.partial: writing chunk 0
2024/04/30 23:45:28 DEBUG : mkv.mkv.caxuzik9.partial: writing chunk 11
2024/04/30 23:45:28 DEBUG : mkv.mkv.caxuzik9.partial: writing chunk 9
2024/04/30 23:46:00 INFO  : Signal received: interrupt
2024/04/30 23:46:00 INFO  : mkv.mkv.caxuzik9.partial: Removing failed copy
2024/04/30 23:46:00 DEBUG : mkv.mkv: multi-thread copy: cancelling transfer on exit
2024/04/30 23:46:00 DEBUG : mkv.mkv: Reopening on read failure after offset 249023 bytes: retry 1/10: context canceled
2024/04/30 23:46:00 DEBUG : mkv.mkv: Reopen failed after offset 249023 bytes read: Get "https://redacted": context canceled
2024/04/30 23:46:00 DEBUG : mkv.mkv: multi-thread copy: chunk 10/12 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:46:00 DEBUG : mkv.mkv: Reopening on read failure after offset 12288 bytes: retry 1/10: context canceled
2024/04/30 23:46:00 DEBUG : mkv.mkv: Reopening on read failure after offset 2024476 bytes: retry 1/10: context canceled
2024/04/30 23:46:00 DEBUG : mkv.mkv: multi-thread copy: chunk 4/12 failed: multi-thread copy: failed to open source: Get "https://redacted": context canceled
2024/04/30 23:46:00 DEBUG : mkv.mkv: multi-thread copy: chunk 9/12 failed: multi-thread copy: failed to open source: Get "https://redacted": context canceled
2024/04/30 23:46:00 DEBUG : mkv.mkv: multi-thread copy: chunk 8/12 failed: multi-thread copy: failed to open source: Get "https://redacted": context canceled
2024/04/30 23:46:00 DEBUG : mkv.mkv: multi-thread copy: chunk 5/12 failed: multi-thread copy: failed to open source: Get "https://redacted": context canceled
2024/04/30 23:46:00 DEBUG : mkv.mkv: multi-thread copy: chunk 6/12 failed: multi-thread copy: failed to open source: Get "https://redacted": context canceled
2024/04/30 23:46:00 DEBUG : mkv.mkv: Reopening on read failure after offset 2417 bytes: retry 1/10: context canceled
2024/04/30 23:46:00 DEBUG : mkv.mkv: Reopening on read failure after offset 11169 bytes: retry 1/10: context canceled
2024/04/30 23:46:00 DEBUG : mkv.mkv: Reopening on read failure after offset 113764861 bytes: retry 1/10: context canceled
2024/04/30 23:46:00 DEBUG : mkv.mkv: Reopening on read failure after offset 114683716 bytes: retry 1/10: context canceled
2024/04/30 23:46:00 DEBUG : mkv.mkv: Reopen failed after offset 12288 bytes read: Get "https://redacted": context canceled
2024/04/30 23:46:00 DEBUG : mkv.mkv: multi-thread copy: chunk 7/12 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:46:00 DEBUG : mkv.mkv: multi-thread copy: abort failed: multi-thread copy: failed to find temp file when aborting chunk writer: object not found
2024/04/30 23:46:00 INFO  : Exiting...
Transferred:   	  219.625 MiB / 23.628 GiB, 1%, 6.468 MiB/s, ETA 1h1m46s
Transferred:            0 / 1, 0%
Elapsed time:        35.6s
Transferring:
 * mkv.mkv:  0% /23.628Gi, 6.468Mi/s, 1h1m46s%

Nah, even worst performance. How about through a local HTTP proxy? Would it help?

sunnylo@SunnyLos-Laptop Downloads % https_proxy=localhost:9090 rclone copy -P -vv --multi-thread-chunk-size 2G --multi-thread-streams 16 "od-seed:/Files/mkv.mkv" .
2024/04/30 23:48:04 DEBUG : rclone: Version "v1.66.0" starting with parameters ["/Users/sunnylo/Documents/bin/rclone" "--multi-thread-chunk-size" "2G" "--multi-thread-streams" "16" "copy" "-P" "-vv" "--multi-thread-chunk-size" "2G" "--multi-thread-streams" "16" "od-seed:/Files/mkv.mkv" "."]
2024/04/30 23:48:04 DEBUG : Creating backend with remote "od-seed:/Files/mkv.mkv"
2024/04/30 23:48:04 DEBUG : Using config file from "/Users/sunnylo/.config/rclone/rclone.conf"
2024/04/30 23:48:05 DEBUG : fs cache: adding new entry for parent of "od-seed:/Files/mkv.mkv", "od-seed:Files"
2024/04/30 23:48:05 DEBUG : Creating backend with remote "."
2024/04/30 23:48:05 DEBUG : fs cache: renaming cache item "." to be canonical "/Users/sunnylo/Downloads"
2024/04/30 23:48:06 DEBUG : mkv.mkv: Need to transfer - File not found at Destination
2024/04/30 23:48:06 DEBUG : mkv.mkv: multi-thread copy: disabling buffering because destination uses OpenWriterAt
2024/04/30 23:48:06 DEBUG : mkv.mkv: multi-thread copy: write buffer set to 131072
2024/04/30 23:48:06 DEBUG : mkv.mkv: multi-thread copy: number of streams 16 was bigger than number of chunks 12
2024/04/30 23:48:06 DEBUG : mkv.mkv: Starting multi-thread copy with 12 chunks of size 2Gi with 12 parallel streams
2024/04/30 23:48:06 DEBUG : mkv.mkv: multi-thread copy: chunk 12/12 (23622320128-25370737554) size 1.628Gi starting
2024/04/30 23:48:06 DEBUG : mkv.mkv: multi-thread copy: chunk 5/12 (8589934592-10737418240) size 2Gi starting
2024/04/30 23:48:06 DEBUG : mkv.mkv: multi-thread copy: chunk 3/12 (4294967296-6442450944) size 2Gi starting
2024/04/30 23:48:06 DEBUG : mkv.mkv: multi-thread copy: chunk 6/12 (10737418240-12884901888) size 2Gi starting
2024/04/30 23:48:06 DEBUG : mkv.mkv: multi-thread copy: chunk 7/12 (12884901888-15032385536) size 2Gi starting
2024/04/30 23:48:06 DEBUG : mkv.mkv: multi-thread copy: chunk 8/12 (15032385536-17179869184) size 2Gi starting
2024/04/30 23:48:06 DEBUG : mkv.mkv: multi-thread copy: chunk 9/12 (17179869184-19327352832) size 2Gi starting
2024/04/30 23:48:06 DEBUG : mkv.mkv: multi-thread copy: chunk 10/12 (19327352832-21474836480) size 2Gi starting
2024/04/30 23:48:06 DEBUG : mkv.mkv: multi-thread copy: chunk 11/12 (21474836480-23622320128) size 2Gi starting
2024/04/30 23:48:06 DEBUG : mkv.mkv: multi-thread copy: chunk 2/12 (2147483648-4294967296) size 2Gi starting
2024/04/30 23:48:06 DEBUG : mkv.mkv: multi-thread copy: chunk 1/12 (0-2147483648) size 2Gi starting
2024/04/30 23:48:06 DEBUG : mkv.mkv: multi-thread copy: chunk 4/12 (6442450944-8589934592) size 2Gi starting
2024/04/30 23:48:07 DEBUG : mkv.mkv.xukudir2.partial: writing chunk 11
2024/04/30 23:48:07 DEBUG : mkv.mkv.xukudir2.partial: writing chunk 5
2024/04/30 23:48:07 DEBUG : mkv.mkv.xukudir2.partial: writing chunk 8
2024/04/30 23:48:07 DEBUG : mkv.mkv.xukudir2.partial: writing chunk 4
2024/04/30 23:48:08 DEBUG : mkv.mkv.xukudir2.partial: writing chunk 0
2024/04/30 23:48:08 DEBUG : mkv.mkv.xukudir2.partial: writing chunk 3
2024/04/30 23:48:08 DEBUG : mkv.mkv.xukudir2.partial: writing chunk 1
2024/04/30 23:48:08 DEBUG : mkv.mkv.xukudir2.partial: writing chunk 6
2024/04/30 23:48:08 DEBUG : mkv.mkv.xukudir2.partial: writing chunk 7
2024/04/30 23:48:08 DEBUG : mkv.mkv.xukudir2.partial: writing chunk 10
2024/04/30 23:48:08 DEBUG : mkv.mkv.xukudir2.partial: writing chunk 9
2024/04/30 23:48:08 DEBUG : mkv.mkv.xukudir2.partial: writing chunk 2
2024/04/30 23:48:28 INFO  : Signal received: interrupt
2024/04/30 23:48:28 INFO  : mkv.mkv.xukudir2.partial: Removing failed copy
2024/04/30 23:48:28 DEBUG : mkv.mkv: multi-thread copy: cancelling transfer on exit
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopening on read failure after offset 99856384 bytes: retry 1/10: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopen failed after offset 99856384 bytes read: Get "https://redacted": context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: multi-thread copy: chunk 2/12 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: multi-thread copy: chunk 12/12 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopening on read failure after offset 148189184 bytes: retry 1/10: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: multi-thread copy: chunk 3/12 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopening on read failure after offset 153169920 bytes: retry 1/10: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopening on read failure after offset 96661504 bytes: retry 1/10: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: multi-thread copy: chunk 1/12 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopening on read failure after offset 105345024 bytes: retry 1/10: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopening on read failure after offset 141520896 bytes: retry 1/10: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopening on read failure after offset 120614912 bytes: retry 1/10: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopening on read failure after offset 136753152 bytes: retry 1/10: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopening on read failure after offset 85471232 bytes: retry 1/10: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopen failed after offset 148189184 bytes read: Get "https://redacted": context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: multi-thread copy: chunk 4/12 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopen failed after offset 153169920 bytes read: Get "https://redacted": context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: multi-thread copy: chunk 11/12 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopen failed after offset 96661504 bytes read: Get "https://redacted": context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: multi-thread copy: chunk 10/12 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopen failed after offset 105345024 bytes read: Get "https://redacted": context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: multi-thread copy: chunk 7/12 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopen failed after offset 141520896 bytes read: Get "https://redacted": context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: multi-thread copy: chunk 8/12 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopen failed after offset 120614912 bytes read: Get "https://redacted": context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: multi-thread copy: chunk 6/12 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopen failed after offset 136753152 bytes read: Get "https://redacted": context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: multi-thread copy: chunk 9/12 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: Reopen failed after offset 85471232 bytes read: Get "https://redacted": context canceled
2024/04/30 23:48:28 DEBUG : mkv.mkv: multi-thread copy: chunk 5/12 failed: multi-thread copy: failed to write chunk: context canceled
2024/04/30 23:48:29 DEBUG : mkv.mkv: multi-thread copy: abort failed: multi-thread copy: failed to find temp file when aborting chunk writer: object not found
2024/04/30 23:48:29 INFO  : Exiting...
Transferred:   	    1.341 GiB / 23.628 GiB, 6%, 60.184 MiB/s, ETA 6m19s
Transferred:            0 / 1, 0%
Elapsed time:        24.7s
Transferring:
 * mkv.mkv:  5% /23.628Gi, 60.180Mi/s, 6m19s%

Wait, what? How come it goes up to 60MiB/s? It's just a proxy? Why would it boost my speed? Anyone could explain? I can do further testing on it and see how to improve the speed.

And for those who may asked, I am using ProxyMan (not affiliated), which it's default port is 9090. I normally use it to play CTF challenges, so

asdffdsa · April 30, 2024, 4:55pm

can you post rclone config redacted od-seed:

jcastill · April 30, 2024, 5:46pm

Based on my experience, OneDrive is overall slow. I've noticed IPv4 is way more efficient than IPv6, so you might be able to try that if you have IPv6 enabled.

Otherwise, even with 10gbps servers I do not get over 500mbps. So I believe this is an artificial limitation on Microsoft end to ease congestion.

SunnyLo · May 1, 2024, 2:12am

Sure of course

% rclone config redacted od-seed:
[od-seed]
type = onedrive
client_id = XXX
client_secret = XXX
auth_url = https://login.microsoftonline.com/redacted/oauth2/v2.0/authorize
token_url = https://login.microsoftonline.com/redacted/oauth2/v2.0/token
token = XXX
drive_id = XXX
drive_type = business
chunk_size = 50M

SunnyLo · May 1, 2024, 2:13am

I don't think I have a IPv6 address, I only got IPv4

% telnet -6 ipv6.telnetmyip.com
Trying 2600:1f16:227:6200::100...
telnet: connect to address 2600:1f16:227:6200::100: Network is unreachable
telnet: Unable to connect to remote host
% curl https://api.ipify.org
XXX.XXX.XXX.XX

jcastill · May 1, 2024, 4:42pm

you can try making the chunk_size bigger. I think maximum is chunk_size = 160Mi

this might help if you are transferring big files.

SunnyLo · May 2, 2024, 3:06am

Tried, even with the maximum 250M, however still no help

jcastill · May 2, 2024, 4:14am

You can try testing your speed to Azure network which is shared with OneDrive... But it sounds like it's just a network problem and not much you can do then.

system · June 1, 2024, 4:14am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.