Very slow upload speeds with onedrive

What is the problem you are having with rclone?

Slow upload speeds with onedrive

Run the command 'rclone version' and share the full output of the command.

rclone v1.61.1

  • os/version: ubuntu 22.04 (64 bit)
  • os/kernel: 6.1.5-060105-generic-64k (aarch64)
  • os/type: linux
  • os/arch: arm64
  • go/version: go1.19.4
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

onedrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount odcrypt-movies: /media/movies 

The rclone config contents with secrets removed.

[odmovies]
type = onedrive
client_id = 
client_secret = 
auth_url = 
token_url = 
drive_id = 
drive_type = business
token = {access token}

[odcrypt-movies]
type = crypt
remote = odmovies:movies
filename_encryption = standard
password = 
password2 = 
directory_name_encryption = true

A log from the command with the -vv flag

2023/01/12 18:40:04 INFO  : 
Transferred:   	      600 MiB / 17.551 GiB, 3%, 2.701 MiB/s, ETA 1h47m12s
Deleted:                0 (files), 2 (dirs)
Transferred:            0 / 1, 0%
Elapsed time:      16m0.3s
Transferring:
 * filedir/filename.extension:  3% /17.551Gi, 2.701Mi/s, 1h47m12s

2023/01/12 18:40:07 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 0, uploading 1, total size 17.551Gi (was 17.551Gi)

I'm using animosity22's homescripts (modified to work with onedrive of course) for the mount script and I enabled both --stats 1m and --vv when running the command.

Am I doing something wrong or is this actually the max speed when uploading to onedrive? It seems to end up around 5Mi/s. I'm using a vps with 1gbit up so it's not a speed issue and the latest ubuntu kernel since I read that could be an issue.

hello and welcome to the forunm

that log, what are the exact commands?

that is the exact command? so not using vfs cache, not using log file?

this is the systemd unit that runs at startup for the rclone mount

[Unit]
Description=RClone Service Onedrive Movies
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
Environment=RCLONE_CONFIG=/home/ubuntu/.config/rclone/rclone.conf
RestartSec=5
ExecStart=/usr/bin/rclone mount odcrypt-movies: /media/movies \
# This is for allowing users other than the user running rclone access to the mount
--allow-other \
# Dropbox is a polling remote so this value can be set very high and any changes are detected via polling.
--dir-cache-time 72h \
# Log file location
--log-file=/home/ubuntu/rclone/logs/onedrive-movies.log \
# Set the log level
--log-level INFO \
# This is setting the file permission on the mount to user and group have the same access and other can read
--umask 002 \
# This sets up the remote control daemon so you can issue rc commands locally
--rc \
# This is the default port it runs on
--rc-addr 127.0.0.1:5574 \
# no-auth is used as no one else uses my server and it is not a shared seedbox
--rc-no-auth \
# The local disk used for caching
--cache-dir=/cache2/Movies \
# This is used for caching files to local disk for streaming
--vfs-cache-mode full \
# This limits the cache size to the value below
--vfs-cache-max-size 100G \
# Wait before uploading
--vfs-write-back 5m \
# This limits the age in the cache if the size is reached and it removes the oldest files first
--vfs-cache-max-age 24h \
# Set the tpslimit
#--tpslimit 12 \
# Set the tpslimit-burst
#--tpslimit-burst 12 \
# Fix IPV6 Errors if you are not running IPV6
#--bind 192.168.1.30 \
# Disable HTTP2
--disable-http2 \
#show upload progress
--stats 1m
#verbose
--vv
ExecStop=/bin/fusermount -uz /media/movies
ExecStartPost=/usr/bin/rclone rc vfs/refresh recursive=true --url 127.0.0.1:5574 _async=true
Restart=on-failure
User=ubuntu
Group=ubuntu

[Install]
WantedBy=multi-user.target

Transferred: 600 MiB / 17.551 GiB, 3%, 2.701 MiB/s, ETA 1h47m12s
ok, perhaps i am confused but the log you posted,
looks like the output from rclone copy/move/sync, not rclone mount

how are you uploading files, using rclone mount or rclone copy/sync/move or what?

onedrive is well known for slow speeds and lots of throttling, and often discussed in the forum.
I will try to find that for you.

ah sorry, like i said im using rclone mount and i had the --stats 1m flag active too but i realized i had set the -vv flag wrong, so here is the log again with--stats 1m removed and only -vv:

2023/01/12 19:42:15 DEBUG : rclone: Version "v1.61.1" starting with parameters ["/usr/bin/rclone" "mount" "odcrypt-tv:" "/media/tv" "--allow-other" "--dir-cache-time" "72h" "--log-file=/home/ubuntu/rclone/logs/onedrive-tv.log" "--umask" "002" "--rc" "--rc-addr" "127.0.0.1:5575" "--rc-no-auth" "--cache-dir=/cache/tv" "--vfs-cache-mode" "full" "--vfs-cache-max-size" "100G" "--vfs-write-back" "5m" "--vfs-cache-max-age" "24h" "--disable-http2" "-vv"]
2023/01/12 19:42:15 NOTICE: Serving remote control on http://127.0.0.1:5575/
2023/01/12 19:42:15 DEBUG : Creating backend with remote "odcrypt-tv:"
2023/01/12 19:42:15 DEBUG : Using config file from "/home/ubuntu/.config/rclone/rclone.conf"
2023/01/12 19:42:15 DEBUG : Creating backend with remote "odtv:tv"
2023/01/12 19:42:19 DEBUG : OneDrive root 'tv': Next delta token is: MzslMjM0OyUyMzE7MzsyYzYxNDhmNC04MmVmLTQyYTEtODViZC0wNzI5ZDhiYmIyNWE7NjM4MDkxNDkzMzk2NTcwMDAwOzI4OTAxODc1MDslMjM7JTIzOyUyMzA
2023/01/12 19:42:19 DEBUG : vfs cache: root is "/cache/tv"
2023/01/12 19:42:19 DEBUG : vfs cache: data root is "/cache/tv/vfs/odcrypt-tv"
2023/01/12 19:42:19 DEBUG : vfs cache: metadata root is "/cache/tv/vfsMeta/odcrypt-tv"
2023/01/12 19:42:19 DEBUG : Creating backend with remote "/cache/tv/vfs/odcrypt-tv/"
2023/01/12 19:42:19 DEBUG : fs cache: renaming cache item "/cache/tv/vfs/odcrypt-tv/" to be canonical "/cache/tv/vfs/odcrypt-tv"
2023/01/12 19:42:19 DEBUG : Creating backend with remote "/cache/tv/vfsMeta/odcrypt-tv/"
2023/01/12 19:42:19 DEBUG : fs cache: renaming cache item "/cache/tv/vfsMeta/odcrypt-tv/" to be canonical "/cache/tv/vfsMeta/odcrypt-tv"
2023/01/12 19:42:19 INFO  : batman test: Removing directory
2023/01/12 19:42:19 INFO  : batman test: Removing directory
2023/01/12 19:42:19 DEBUG : Encrypted drive 'odcrypt-tv:': Mounting on "/media/tv"
2023/01/12 19:42:19 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2023/01/12 19:42:19 DEBUG : : Root: 
2023/01/12 19:42:19 DEBUG : : >Root: node=/, err=<nil>
2023/01/12 19:42:19 DEBUG : rc: "vfs/refresh": with parameters map[_async:true recursive:true]
2023/01/12 19:42:19 DEBUG : rc: "vfs/refresh": reply map[jobid:1]: <nil>
2023/01/12 19:42:19 DEBUG : : Reading directory tree
2023/01/12 19:42:21 DEBUG : /: Attr: 
2023/01/12 19:42:21 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:42:22 DEBUG : : Reading directory tree done in 2.383422706s
2023/01/12 19:43:17 DEBUG : /: Attr: 
2023/01/12 19:43:17 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:17 DEBUG : /: ReadDirAll: 
2023/01/12 19:43:17 DEBUG : /: >ReadDirAll: item=4, err=<nil>
2023/01/12 19:43:17 DEBUG : /: Lookup: name="batman test"
2023/01/12 19:43:17 DEBUG : /: >Lookup: node=batman test/, err=<nil>
2023/01/12 19:43:17 DEBUG : batman test/: Attr: 
2023/01/12 19:43:17 DEBUG : batman test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:17 DEBUG : batman test/: ReadDirAll: 
2023/01/12 19:43:17 DEBUG : batman test/: >ReadDirAll: item=2, err=<nil>
2023/01/12 19:43:17 DEBUG : /: Lookup: name="media"
2023/01/12 19:43:17 DEBUG : /: >Lookup: node=media/, err=<nil>
2023/01/12 19:43:17 DEBUG : media/: Attr: 
2023/01/12 19:43:17 DEBUG : media/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:17 DEBUG : media/: ReadDirAll: 
2023/01/12 19:43:17 DEBUG : media/: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:43:17 DEBUG : media/: Lookup: name="tv"
2023/01/12 19:43:17 DEBUG : media/: >Lookup: node=media/tv/, err=<nil>
2023/01/12 19:43:17 DEBUG : media/tv/: Attr: 
2023/01/12 19:43:17 DEBUG : media/tv/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:17 DEBUG : media/tv/: ReadDirAll: 
2023/01/12 19:43:17 DEBUG : media/tv/: >ReadDirAll: item=2, err=<nil>
2023/01/12 19:43:19 DEBUG : OneDrive root 'tv': Checking for changes on remote
2023/01/12 19:43:19 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2023/01/12 19:43:41 DEBUG : /: Attr: 
2023/01/12 19:43:41 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:41 DEBUG : /: ReadDirAll: 
2023/01/12 19:43:41 DEBUG : /: >ReadDirAll: item=4, err=<nil>
2023/01/12 19:43:43 DEBUG : /: Lookup: name="batman test"
2023/01/12 19:43:43 DEBUG : /: >Lookup: node=batman test/, err=<nil>
2023/01/12 19:43:43 DEBUG : batman test/: Attr: 
2023/01/12 19:43:43 DEBUG : batman test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:43 DEBUG : batman test/: ReadDirAll: 
2023/01/12 19:43:43 DEBUG : batman test/: >ReadDirAll: item=2, err=<nil>
2023/01/12 19:43:53 DEBUG : /: Lookup: name="media"
2023/01/12 19:43:53 DEBUG : /: >Lookup: node=media/, err=<nil>
2023/01/12 19:43:53 DEBUG : media/: Attr: 
2023/01/12 19:43:53 DEBUG : media/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:53 DEBUG : media/: ReadDirAll: 
2023/01/12 19:43:53 DEBUG : media/: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:43:53 DEBUG : media/: Lookup: name="tv"
2023/01/12 19:43:53 DEBUG : media/: >Lookup: node=media/tv/, err=<nil>
2023/01/12 19:43:53 DEBUG : media/tv/: Attr: 
2023/01/12 19:43:53 DEBUG : media/tv/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:53 DEBUG : media/tv/: ReadDirAll: 
2023/01/12 19:43:53 DEBUG : media/tv/: >ReadDirAll: item=2, err=<nil>
2023/01/12 19:43:58 DEBUG : /: Lookup: name="media"
2023/01/12 19:43:58 DEBUG : /: >Lookup: node=media/, err=<nil>
2023/01/12 19:43:58 DEBUG : media/: Attr: 
2023/01/12 19:43:58 DEBUG : media/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:58 DEBUG : media/: ReadDirAll: 
2023/01/12 19:43:58 DEBUG : media/: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:43:58 DEBUG : media/: Lookup: name="tv"
2023/01/12 19:43:58 DEBUG : media/: >Lookup: node=media/tv/, err=<nil>
2023/01/12 19:43:58 DEBUG : media/tv/: Attr: 
2023/01/12 19:43:58 DEBUG : media/tv/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:58 DEBUG : media/: Attr: 
2023/01/12 19:43:58 DEBUG : media/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:58 DEBUG : /: Attr: 
2023/01/12 19:43:58 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:58 DEBUG : /: ReadDirAll: 
2023/01/12 19:43:58 DEBUG : /: >ReadDirAll: item=4, err=<nil>
2023/01/12 19:43:58 DEBUG : /: Lookup: name="batman test"
2023/01/12 19:43:58 DEBUG : /: >Lookup: node=batman test/, err=<nil>
2023/01/12 19:43:58 DEBUG : batman test/: Attr: 
2023/01/12 19:43:58 DEBUG : batman test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:58 DEBUG : /: Remove: name="media"
2023/01/12 19:43:58 ERROR : media/: Dir.Remove not empty
2023/01/12 19:43:58 DEBUG : /: >Remove: err=directory not empty
2023/01/12 19:43:58 DEBUG : media/: ReadDirAll: 
2023/01/12 19:43:58 DEBUG : media/: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:43:58 DEBUG : media/: Remove: name="tv"
2023/01/12 19:43:58 DEBUG : media/: Attr: 
2023/01/12 19:43:58 DEBUG : media/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:58 DEBUG : batman test/: ReadDirAll: 
2023/01/12 19:43:58 DEBUG : batman test/: >ReadDirAll: item=2, err=<nil>
2023/01/12 19:44:00 DEBUG : media: Added virtual directory entry vDel: "tv"
2023/01/12 19:44:00 DEBUG : media/: >Remove: err=<nil>
2023/01/12 19:44:00 DEBUG : media/: ReadDirAll: 
2023/01/12 19:44:00 DEBUG : media/: >ReadDirAll: item=2, err=<nil>
2023/01/12 19:44:00 DEBUG : /: Lookup: name="media"
2023/01/12 19:44:00 DEBUG : /: >Lookup: node=media/, err=<nil>
2023/01/12 19:44:00 DEBUG : media/: Attr: 
2023/01/12 19:44:00 DEBUG : media/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:44:00 DEBUG : /: Remove: name="media"
2023/01/12 19:44:00 DEBUG : /: Attr: 
2023/01/12 19:44:00 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:44:01 DEBUG : : Added virtual directory entry vDel: "media"
2023/01/12 19:44:01 DEBUG : /: >Remove: err=<nil>
2023/01/12 19:44:01 DEBUG : /: ReadDirAll: 
2023/01/12 19:44:01 DEBUG : /: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:44:01 DEBUG : /: Lookup: name="batman test"
2023/01/12 19:44:01 DEBUG : /: >Lookup: node=batman test/, err=<nil>
2023/01/12 19:44:01 DEBUG : batman test/: Attr: 
2023/01/12 19:44:01 DEBUG : batman test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:44:01 DEBUG : batman test/: ReadDirAll: 
2023/01/12 19:44:01 DEBUG : batman test/: >ReadDirAll: item=2, err=<nil>
2023/01/12 19:44:04 DEBUG : /: Lookup: name="batman test"
2023/01/12 19:44:04 DEBUG : /: >Lookup: node=batman test/, err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/: Attr: 
2023/01/12 19:44:04 DEBUG : batman test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/: Lookup: name="The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/: >Lookup: node=<nil>, err=no such file or directory
2023/01/12 19:44:04 DEBUG : batman test/: Lookup: name="The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/: >Lookup: node=<nil>, err=no such file or directory
2023/01/12 19:44:04 DEBUG : batman test/: Lookup: name="The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/: >Lookup: node=<nil>, err=no such file or directory
2023/01/12 19:44:04 DEBUG : batman test/: Lookup: name="The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/: >Lookup: node=<nil>, err=no such file or directory
2023/01/12 19:44:04 DEBUG : batman test/: Lookup: name="The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/: >Lookup: node=<nil>, err=no such file or directory
2023/01/12 19:44:04 DEBUG : batman test/: Lookup: name="The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/: >Lookup: node=<nil>, err=no such file or directory
2023/01/12 19:44:04 DEBUG : batman test/: Create: name="The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: Open: flags=O_WRONLY|O_CREATE|0x20000
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: newRWFileHandle: 
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv(0x4001d952c0): openPending: 
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: vfs cache: truncate to size=0 (not needed as size correct)
2023/01/12 19:44:04 DEBUG : batman test: Added virtual directory entry vAddFile: "The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv(0x4001d952c0): >openPending: err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: >newRWFileHandle: err=<nil>
2023/01/12 19:44:04 DEBUG : batman test: Added virtual directory entry vAddFile: "The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: >Open: fd=batman test/The Batman (2022) Bluray-1080p.mkv (rw), err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/: >Create: node=batman test/The Batman (2022) Bluray-1080p.mkv, handle=&{batman test/The Batman (2022) Bluray-1080p.mkv (rw)}, err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: Attr: 
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: >Attr: a=valid=1s ino=0 size=0 mode=-rw-rw-r--, err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: Setattr: a=Setattr [ID=0xbc Node=0x3 Uid=1001 Gid=1001 Pid=2254] size=0 handle=0x1 lockowner
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: Truncating 1 file handles
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: >Setattr: err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: Attr: 
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: >Attr: a=valid=1s ino=0 size=0 mode=-rw-rw-r--, err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: Setattr: a=Setattr [ID=0xbe Node=0x3 Uid=1001 Gid=1001 Pid=2254] mode=-rw-rw-r-- handle=INVALID-0x0
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: >Setattr: err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: Attr: 
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: >Attr: a=valid=1s ino=0 size=0 mode=-rw-rw-r--, err=<nil>
2023/01/12 19:44:04 DEBUG : &{batman test/The Batman (2022) Bluray-1080p.mkv (rw)}: Write: len=131072, offset=0
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv(0x4001d952c0): _writeAt: size=131072, off=0
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv(0x4001d952c0): >_writeAt: n=131072, err=<nil>
2023/01/12 19:44:04 DEBUG : &{batman test/The Batman (2022) Bluray-1080p.mkv (rw)}: >Write: written=131072, err=<nil>
2023/01/12 19:44:04 DEBUG : &{batman test/The Batman (2022) Bluray-1080p.mkv (rw)}: Write: len=131072, offset=131072

it must have bugged out somehow, for some reasonit repeated the last 4 lines for 600k lines.
log file after it started uploading

2023/01/12 19:50:47 DEBUG : /: Attr: 
2023/01/12 19:50:47 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:50:47 DEBUG : /: ReadDirAll: 
2023/01/12 19:50:47 DEBUG : /: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:50:47 DEBUG : /: Lookup: name="test"
2023/01/12 19:50:47 DEBUG : /: >Lookup: node=test/, err=<nil>
2023/01/12 19:50:47 DEBUG : test/: Attr: 
2023/01/12 19:50:47 DEBUG : test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:50:47 DEBUG : test/: ReadDirAll: 
2023/01/12 19:50:47 DEBUG : test/: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:50:48 DEBUG : test/test file.mkv: vfs cache: starting upload
2023/01/12 19:50:48 DEBUG : 960umddq1caht7q7kilcdq5hqk/6v17fnpvv03c65aa1ns0am1i444i4bbcpff5r9jt2sst64f7l29issggr1ef513sbuh77kel8mv9k: Starting multipart upload
2023/01/12 19:50:49 DEBUG : 960umddq1caht7q7kilcdq5hqk/6v17fnpvv03c65aa1ns0am1i444i4bbcpff5r9jt2sst64f7l29issggr1ef513sbuh77kel8mv9k: Uploading segment 0/18850321057 size 10485760
2023/01/12 19:50:53 DEBUG : 960umddq1caht7q7kilcdq5hqk/6v17fnpvv03c65aa1ns0am1i444i4bbcpff5r9jt2sst64f7l29issggr1ef513sbuh77kel8mv9k: Uploading segment 10485760/18850321057 size 10485760
2023/01/12 19:50:53 DEBUG : /: Attr: 
2023/01/12 19:50:53 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:50:53 DEBUG : /: ReadDirAll: 
2023/01/12 19:50:53 DEBUG : /: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:50:53 DEBUG : /: Lookup: name="test"
2023/01/12 19:50:53 DEBUG : /: >Lookup: node=test/, err=<nil>
2023/01/12 19:50:53 DEBUG : test/: Attr: 
2023/01/12 19:50:53 DEBUG : test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:50:53 DEBUG : test/: ReadDirAll: 
2023/01/12 19:50:53 DEBUG : test/: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:50:55 DEBUG : 960umddq1caht7q7kilcdq5hqk/6v17fnpvv03c65aa1ns0am1i444i4bbcpff5r9jt2sst64f7l29issggr1ef513sbuh77kel8mv9k: Uploading segment 20971520/18850321057 size 10485760

ok, but the command in the systemd file, does not seem to match the command from that log, correct?

in any event, might try

  • increasing --transfers
  • add --buffer-size=0

no bug, that is to be expected.

some app, perhaps media server, such as plex is asking rclone to iterate over the dir/files.
or
ExecStartPost command does that.

600K lines divided by 4 = 250K files. and that might be a lot of files for onedrive, which has lots of throttling.

tried with transfer=6 and buffer-size=0, neither seemed to affect the upload speed, is they anything else i could try?

might try tweaking --onedrive-chunk-size

perhaps @Ole has a suggestion?

We have nothing to do here. It's OneDrive limit which is the reason people prefer GG Drive than OneDrive. On the same machine, GG Drive upload speed 80-165MB/s, OneDrive is just 4-7.5MB/s.
You might search on Reddit or other forums, they all reported the issue long time ago and no fix until now.

The config looks good to me, I don’t think there is anything we can do to make it faster.

Agree, the speed of OneDrive seems to depend on the subscription plan and price. The more you pay the faster it is.

OneDrive also sometimes slows down the upload of large files, especially if you already uploaded a lot within the past 24 hours.

It looks like an intentional business decision to me.

try doing an rclone sync with --transfers=1 flag on the single large file you're talking about. see what the bandwidth is then. as one of the resident onedrive complainers on this forum, you should be able to get 50mbps on both personal and business onedrive's with a single file transfer, assuming you're within a normal latency perimeter of the data center.

jared

tried that just now, I'm still getting ~5Mi/s, so I guess it's some other issue, maybe I'm just too far away from the data center like you said.

oh well, looks like I'll have to find another way to store my content. But thanks for the help everyone, I really appreciate it.

That is roughly the same as this:

The first is counting bytes transferred by rclone, the second counting bits on the transmission line. The ratio is roughly 10.

thanks @Ole , almost missed that. yeah I think you're getting everything that Microsoft graph gives for single file transfer. I don't think that's too bad actually, for what I do with it :slight_smile: I guess if you're dealing single large files and trying to save space on the local disk by using rclone mount, I can see how you'd want more. however, at that point I don't see how double or even quadruple of that would be enough. probably time for a local storage server.

Ah yeah you're right, forgot about the different unit there for a sec.

I would, but this is all on a vps haha

thanks for all the help guys :slight_smile: