Very slow upload speeds with onedrive

rasmaplasma · January 12, 2023, 6:49pm

What is the problem you are having with rclone?

Slow upload speeds with onedrive

Run the command 'rclone version' and share the full output of the command.

rclone v1.61.1

os/version: ubuntu 22.04 (64 bit)
os/kernel: 6.1.5-060105-generic-64k (aarch64)
os/type: linux
os/arch: arm64
go/version: go1.19.4
go/linking: static
go/tags: none

Which cloud storage system are you using? (eg Google Drive)

onedrive

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone mount odcrypt-movies: /media/movies

The rclone config contents with secrets removed.

[odmovies]
type = onedrive
client_id = 
client_secret = 
auth_url = 
token_url = 
drive_id = 
drive_type = business
token = {access token}

[odcrypt-movies]
type = crypt
remote = odmovies:movies
filename_encryption = standard
password = 
password2 = 
directory_name_encryption = true

A log from the command with the `-vv` flag

2023/01/12 18:40:04 INFO  : 
Transferred:   	      600 MiB / 17.551 GiB, 3%, 2.701 MiB/s, ETA 1h47m12s
Deleted:                0 (files), 2 (dirs)
Transferred:            0 / 1, 0%
Elapsed time:      16m0.3s
Transferring:
 * filedir/filename.extension:  3% /17.551Gi, 2.701Mi/s, 1h47m12s

2023/01/12 18:40:07 INFO  : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 0, uploading 1, total size 17.551Gi (was 17.551Gi)

I'm using animosity22's homescripts (modified to work with onedrive of course) for the mount script and I enabled both --stats 1m and --vv when running the command.

Am I doing something wrong or is this actually the max speed when uploading to onedrive? It seems to end up around 5Mi/s. I'm using a vps with 1gbit up so it's not a speed issue and the latest ubuntu kernel since I read that could be an issue.

asdffdsa · January 12, 2023, 7:07pm

hello and welcome to the forunm

that log, what are the exact commands?

that is the exact command? so not using vfs cache, not using log file?

rasmaplasma · January 12, 2023, 7:15pm

this is the systemd unit that runs at startup for the rclone mount

[Unit]
Description=RClone Service Onedrive Movies
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
Environment=RCLONE_CONFIG=/home/ubuntu/.config/rclone/rclone.conf
RestartSec=5
ExecStart=/usr/bin/rclone mount odcrypt-movies: /media/movies \
# This is for allowing users other than the user running rclone access to the mount
--allow-other \
# Dropbox is a polling remote so this value can be set very high and any changes are detected via polling.
--dir-cache-time 72h \
# Log file location
--log-file=/home/ubuntu/rclone/logs/onedrive-movies.log \
# Set the log level
--log-level INFO \
# This is setting the file permission on the mount to user and group have the same access and other can read
--umask 002 \
# This sets up the remote control daemon so you can issue rc commands locally
--rc \
# This is the default port it runs on
--rc-addr 127.0.0.1:5574 \
# no-auth is used as no one else uses my server and it is not a shared seedbox
--rc-no-auth \
# The local disk used for caching
--cache-dir=/cache2/Movies \
# This is used for caching files to local disk for streaming
--vfs-cache-mode full \
# This limits the cache size to the value below
--vfs-cache-max-size 100G \
# Wait before uploading
--vfs-write-back 5m \
# This limits the age in the cache if the size is reached and it removes the oldest files first
--vfs-cache-max-age 24h \
# Set the tpslimit
#--tpslimit 12 \
# Set the tpslimit-burst
#--tpslimit-burst 12 \
# Fix IPV6 Errors if you are not running IPV6
#--bind 192.168.1.30 \
# Disable HTTP2
--disable-http2 \
#show upload progress
--stats 1m
#verbose
--vv
ExecStop=/bin/fusermount -uz /media/movies
ExecStartPost=/usr/bin/rclone rc vfs/refresh recursive=true --url 127.0.0.1:5574 _async=true
Restart=on-failure
User=ubuntu
Group=ubuntu

[Install]
WantedBy=multi-user.target

asdffdsa · January 12, 2023, 7:21pm

Transferred: 600 MiB / 17.551 GiB, 3%, 2.701 MiB/s, ETA 1h47m12s
ok, perhaps i am confused but the log you posted,
looks like the output from rclone copy/move/sync, not rclone mount

how are you uploading files, using rclone mount or rclone copy/sync/move or what?

onedrive is well known for slow speeds and lots of throttling, and often discussed in the forum.
I will try to find that for you.

rasmaplasma · January 12, 2023, 8:01pm

ah sorry, like i said im using rclone mount and i had the --stats 1m flag active too but i realized i had set the -vv flag wrong, so here is the log again with--stats 1m removed and only -vv:

2023/01/12 19:42:15 DEBUG : rclone: Version "v1.61.1" starting with parameters ["/usr/bin/rclone" "mount" "odcrypt-tv:" "/media/tv" "--allow-other" "--dir-cache-time" "72h" "--log-file=/home/ubuntu/rclone/logs/onedrive-tv.log" "--umask" "002" "--rc" "--rc-addr" "127.0.0.1:5575" "--rc-no-auth" "--cache-dir=/cache/tv" "--vfs-cache-mode" "full" "--vfs-cache-max-size" "100G" "--vfs-write-back" "5m" "--vfs-cache-max-age" "24h" "--disable-http2" "-vv"]
2023/01/12 19:42:15 NOTICE: Serving remote control on http://127.0.0.1:5575/
2023/01/12 19:42:15 DEBUG : Creating backend with remote "odcrypt-tv:"
2023/01/12 19:42:15 DEBUG : Using config file from "/home/ubuntu/.config/rclone/rclone.conf"
2023/01/12 19:42:15 DEBUG : Creating backend with remote "odtv:tv"
2023/01/12 19:42:19 DEBUG : OneDrive root 'tv': Next delta token is: MzslMjM0OyUyMzE7MzsyYzYxNDhmNC04MmVmLTQyYTEtODViZC0wNzI5ZDhiYmIyNWE7NjM4MDkxNDkzMzk2NTcwMDAwOzI4OTAxODc1MDslMjM7JTIzOyUyMzA
2023/01/12 19:42:19 DEBUG : vfs cache: root is "/cache/tv"
2023/01/12 19:42:19 DEBUG : vfs cache: data root is "/cache/tv/vfs/odcrypt-tv"
2023/01/12 19:42:19 DEBUG : vfs cache: metadata root is "/cache/tv/vfsMeta/odcrypt-tv"
2023/01/12 19:42:19 DEBUG : Creating backend with remote "/cache/tv/vfs/odcrypt-tv/"
2023/01/12 19:42:19 DEBUG : fs cache: renaming cache item "/cache/tv/vfs/odcrypt-tv/" to be canonical "/cache/tv/vfs/odcrypt-tv"
2023/01/12 19:42:19 DEBUG : Creating backend with remote "/cache/tv/vfsMeta/odcrypt-tv/"
2023/01/12 19:42:19 DEBUG : fs cache: renaming cache item "/cache/tv/vfsMeta/odcrypt-tv/" to be canonical "/cache/tv/vfsMeta/odcrypt-tv"
2023/01/12 19:42:19 INFO  : batman test: Removing directory
2023/01/12 19:42:19 INFO  : batman test: Removing directory
2023/01/12 19:42:19 DEBUG : Encrypted drive 'odcrypt-tv:': Mounting on "/media/tv"
2023/01/12 19:42:19 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2023/01/12 19:42:19 DEBUG : : Root: 
2023/01/12 19:42:19 DEBUG : : >Root: node=/, err=<nil>
2023/01/12 19:42:19 DEBUG : rc: "vfs/refresh": with parameters map[_async:true recursive:true]
2023/01/12 19:42:19 DEBUG : rc: "vfs/refresh": reply map[jobid:1]: <nil>
2023/01/12 19:42:19 DEBUG : : Reading directory tree
2023/01/12 19:42:21 DEBUG : /: Attr: 
2023/01/12 19:42:21 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:42:22 DEBUG : : Reading directory tree done in 2.383422706s
2023/01/12 19:43:17 DEBUG : /: Attr: 
2023/01/12 19:43:17 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:17 DEBUG : /: ReadDirAll: 
2023/01/12 19:43:17 DEBUG : /: >ReadDirAll: item=4, err=<nil>
2023/01/12 19:43:17 DEBUG : /: Lookup: name="batman test"
2023/01/12 19:43:17 DEBUG : /: >Lookup: node=batman test/, err=<nil>
2023/01/12 19:43:17 DEBUG : batman test/: Attr: 
2023/01/12 19:43:17 DEBUG : batman test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:17 DEBUG : batman test/: ReadDirAll: 
2023/01/12 19:43:17 DEBUG : batman test/: >ReadDirAll: item=2, err=<nil>
2023/01/12 19:43:17 DEBUG : /: Lookup: name="media"
2023/01/12 19:43:17 DEBUG : /: >Lookup: node=media/, err=<nil>
2023/01/12 19:43:17 DEBUG : media/: Attr: 
2023/01/12 19:43:17 DEBUG : media/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:17 DEBUG : media/: ReadDirAll: 
2023/01/12 19:43:17 DEBUG : media/: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:43:17 DEBUG : media/: Lookup: name="tv"
2023/01/12 19:43:17 DEBUG : media/: >Lookup: node=media/tv/, err=<nil>
2023/01/12 19:43:17 DEBUG : media/tv/: Attr: 
2023/01/12 19:43:17 DEBUG : media/tv/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:17 DEBUG : media/tv/: ReadDirAll: 
2023/01/12 19:43:17 DEBUG : media/tv/: >ReadDirAll: item=2, err=<nil>
2023/01/12 19:43:19 DEBUG : OneDrive root 'tv': Checking for changes on remote
2023/01/12 19:43:19 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2023/01/12 19:43:41 DEBUG : /: Attr: 
2023/01/12 19:43:41 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:41 DEBUG : /: ReadDirAll: 
2023/01/12 19:43:41 DEBUG : /: >ReadDirAll: item=4, err=<nil>
2023/01/12 19:43:43 DEBUG : /: Lookup: name="batman test"
2023/01/12 19:43:43 DEBUG : /: >Lookup: node=batman test/, err=<nil>
2023/01/12 19:43:43 DEBUG : batman test/: Attr: 
2023/01/12 19:43:43 DEBUG : batman test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:43 DEBUG : batman test/: ReadDirAll: 
2023/01/12 19:43:43 DEBUG : batman test/: >ReadDirAll: item=2, err=<nil>
2023/01/12 19:43:53 DEBUG : /: Lookup: name="media"
2023/01/12 19:43:53 DEBUG : /: >Lookup: node=media/, err=<nil>
2023/01/12 19:43:53 DEBUG : media/: Attr: 
2023/01/12 19:43:53 DEBUG : media/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:53 DEBUG : media/: ReadDirAll: 
2023/01/12 19:43:53 DEBUG : media/: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:43:53 DEBUG : media/: Lookup: name="tv"
2023/01/12 19:43:53 DEBUG : media/: >Lookup: node=media/tv/, err=<nil>
2023/01/12 19:43:53 DEBUG : media/tv/: Attr: 
2023/01/12 19:43:53 DEBUG : media/tv/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:53 DEBUG : media/tv/: ReadDirAll: 
2023/01/12 19:43:53 DEBUG : media/tv/: >ReadDirAll: item=2, err=<nil>
2023/01/12 19:43:58 DEBUG : /: Lookup: name="media"
2023/01/12 19:43:58 DEBUG : /: >Lookup: node=media/, err=<nil>
2023/01/12 19:43:58 DEBUG : media/: Attr: 
2023/01/12 19:43:58 DEBUG : media/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:58 DEBUG : media/: ReadDirAll: 
2023/01/12 19:43:58 DEBUG : media/: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:43:58 DEBUG : media/: Lookup: name="tv"
2023/01/12 19:43:58 DEBUG : media/: >Lookup: node=media/tv/, err=<nil>
2023/01/12 19:43:58 DEBUG : media/tv/: Attr: 
2023/01/12 19:43:58 DEBUG : media/tv/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:58 DEBUG : media/: Attr: 
2023/01/12 19:43:58 DEBUG : media/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:58 DEBUG : /: Attr: 
2023/01/12 19:43:58 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:58 DEBUG : /: ReadDirAll: 
2023/01/12 19:43:58 DEBUG : /: >ReadDirAll: item=4, err=<nil>
2023/01/12 19:43:58 DEBUG : /: Lookup: name="batman test"
2023/01/12 19:43:58 DEBUG : /: >Lookup: node=batman test/, err=<nil>
2023/01/12 19:43:58 DEBUG : batman test/: Attr: 
2023/01/12 19:43:58 DEBUG : batman test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:58 DEBUG : /: Remove: name="media"
2023/01/12 19:43:58 ERROR : media/: Dir.Remove not empty
2023/01/12 19:43:58 DEBUG : /: >Remove: err=directory not empty
2023/01/12 19:43:58 DEBUG : media/: ReadDirAll: 
2023/01/12 19:43:58 DEBUG : media/: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:43:58 DEBUG : media/: Remove: name="tv"
2023/01/12 19:43:58 DEBUG : media/: Attr: 
2023/01/12 19:43:58 DEBUG : media/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:43:58 DEBUG : batman test/: ReadDirAll: 
2023/01/12 19:43:58 DEBUG : batman test/: >ReadDirAll: item=2, err=<nil>
2023/01/12 19:44:00 DEBUG : media: Added virtual directory entry vDel: "tv"
2023/01/12 19:44:00 DEBUG : media/: >Remove: err=<nil>
2023/01/12 19:44:00 DEBUG : media/: ReadDirAll: 
2023/01/12 19:44:00 DEBUG : media/: >ReadDirAll: item=2, err=<nil>
2023/01/12 19:44:00 DEBUG : /: Lookup: name="media"
2023/01/12 19:44:00 DEBUG : /: >Lookup: node=media/, err=<nil>
2023/01/12 19:44:00 DEBUG : media/: Attr: 
2023/01/12 19:44:00 DEBUG : media/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:44:00 DEBUG : /: Remove: name="media"
2023/01/12 19:44:00 DEBUG : /: Attr: 
2023/01/12 19:44:00 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:44:01 DEBUG : : Added virtual directory entry vDel: "media"
2023/01/12 19:44:01 DEBUG : /: >Remove: err=<nil>
2023/01/12 19:44:01 DEBUG : /: ReadDirAll: 
2023/01/12 19:44:01 DEBUG : /: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:44:01 DEBUG : /: Lookup: name="batman test"
2023/01/12 19:44:01 DEBUG : /: >Lookup: node=batman test/, err=<nil>
2023/01/12 19:44:01 DEBUG : batman test/: Attr: 
2023/01/12 19:44:01 DEBUG : batman test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:44:01 DEBUG : batman test/: ReadDirAll: 
2023/01/12 19:44:01 DEBUG : batman test/: >ReadDirAll: item=2, err=<nil>
2023/01/12 19:44:04 DEBUG : /: Lookup: name="batman test"
2023/01/12 19:44:04 DEBUG : /: >Lookup: node=batman test/, err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/: Attr: 
2023/01/12 19:44:04 DEBUG : batman test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/: Lookup: name="The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/: >Lookup: node=<nil>, err=no such file or directory
2023/01/12 19:44:04 DEBUG : batman test/: Lookup: name="The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/: >Lookup: node=<nil>, err=no such file or directory
2023/01/12 19:44:04 DEBUG : batman test/: Lookup: name="The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/: >Lookup: node=<nil>, err=no such file or directory
2023/01/12 19:44:04 DEBUG : batman test/: Lookup: name="The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/: >Lookup: node=<nil>, err=no such file or directory
2023/01/12 19:44:04 DEBUG : batman test/: Lookup: name="The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/: >Lookup: node=<nil>, err=no such file or directory
2023/01/12 19:44:04 DEBUG : batman test/: Lookup: name="The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/: >Lookup: node=<nil>, err=no such file or directory
2023/01/12 19:44:04 DEBUG : batman test/: Create: name="The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: Open: flags=O_WRONLY|O_CREATE|0x20000
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: newRWFileHandle: 
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv(0x4001d952c0): openPending: 
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: vfs cache: truncate to size=0 (not needed as size correct)
2023/01/12 19:44:04 DEBUG : batman test: Added virtual directory entry vAddFile: "The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv(0x4001d952c0): >openPending: err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: >newRWFileHandle: err=<nil>
2023/01/12 19:44:04 DEBUG : batman test: Added virtual directory entry vAddFile: "The Batman (2022) Bluray-1080p.mkv"
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: >Open: fd=batman test/The Batman (2022) Bluray-1080p.mkv (rw), err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/: >Create: node=batman test/The Batman (2022) Bluray-1080p.mkv, handle=&{batman test/The Batman (2022) Bluray-1080p.mkv (rw)}, err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: Attr: 
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: >Attr: a=valid=1s ino=0 size=0 mode=-rw-rw-r--, err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: Setattr: a=Setattr [ID=0xbc Node=0x3 Uid=1001 Gid=1001 Pid=2254] size=0 handle=0x1 lockowner
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: Truncating 1 file handles
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: >Setattr: err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: Attr: 
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: >Attr: a=valid=1s ino=0 size=0 mode=-rw-rw-r--, err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: Setattr: a=Setattr [ID=0xbe Node=0x3 Uid=1001 Gid=1001 Pid=2254] mode=-rw-rw-r-- handle=INVALID-0x0
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: >Setattr: err=<nil>
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: Attr: 
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv: >Attr: a=valid=1s ino=0 size=0 mode=-rw-rw-r--, err=<nil>
2023/01/12 19:44:04 DEBUG : &{batman test/The Batman (2022) Bluray-1080p.mkv (rw)}: Write: len=131072, offset=0
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv(0x4001d952c0): _writeAt: size=131072, off=0
2023/01/12 19:44:04 DEBUG : batman test/The Batman (2022) Bluray-1080p.mkv(0x4001d952c0): >_writeAt: n=131072, err=<nil>
2023/01/12 19:44:04 DEBUG : &{batman test/The Batman (2022) Bluray-1080p.mkv (rw)}: >Write: written=131072, err=<nil>
2023/01/12 19:44:04 DEBUG : &{batman test/The Batman (2022) Bluray-1080p.mkv (rw)}: Write: len=131072, offset=131072

it must have bugged out somehow, for some reasonit repeated the last 4 lines for 600k lines.
log file after it started uploading

2023/01/12 19:50:47 DEBUG : /: Attr: 
2023/01/12 19:50:47 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:50:47 DEBUG : /: ReadDirAll: 
2023/01/12 19:50:47 DEBUG : /: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:50:47 DEBUG : /: Lookup: name="test"
2023/01/12 19:50:47 DEBUG : /: >Lookup: node=test/, err=<nil>
2023/01/12 19:50:47 DEBUG : test/: Attr: 
2023/01/12 19:50:47 DEBUG : test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:50:47 DEBUG : test/: ReadDirAll: 
2023/01/12 19:50:47 DEBUG : test/: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:50:48 DEBUG : test/test file.mkv: vfs cache: starting upload
2023/01/12 19:50:48 DEBUG : 960umddq1caht7q7kilcdq5hqk/6v17fnpvv03c65aa1ns0am1i444i4bbcpff5r9jt2sst64f7l29issggr1ef513sbuh77kel8mv9k: Starting multipart upload
2023/01/12 19:50:49 DEBUG : 960umddq1caht7q7kilcdq5hqk/6v17fnpvv03c65aa1ns0am1i444i4bbcpff5r9jt2sst64f7l29issggr1ef513sbuh77kel8mv9k: Uploading segment 0/18850321057 size 10485760
2023/01/12 19:50:53 DEBUG : 960umddq1caht7q7kilcdq5hqk/6v17fnpvv03c65aa1ns0am1i444i4bbcpff5r9jt2sst64f7l29issggr1ef513sbuh77kel8mv9k: Uploading segment 10485760/18850321057 size 10485760
2023/01/12 19:50:53 DEBUG : /: Attr: 
2023/01/12 19:50:53 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:50:53 DEBUG : /: ReadDirAll: 
2023/01/12 19:50:53 DEBUG : /: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:50:53 DEBUG : /: Lookup: name="test"
2023/01/12 19:50:53 DEBUG : /: >Lookup: node=test/, err=<nil>
2023/01/12 19:50:53 DEBUG : test/: Attr: 
2023/01/12 19:50:53 DEBUG : test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/12 19:50:53 DEBUG : test/: ReadDirAll: 
2023/01/12 19:50:53 DEBUG : test/: >ReadDirAll: item=3, err=<nil>
2023/01/12 19:50:55 DEBUG : 960umddq1caht7q7kilcdq5hqk/6v17fnpvv03c65aa1ns0am1i444i4bbcpff5r9jt2sst64f7l29issggr1ef513sbuh77kel8mv9k: Uploading segment 20971520/18850321057 size 10485760

asdffdsa · January 12, 2023, 8:55pm

ok, but the command in the systemd file, does not seem to match the command from that log, correct?

in any event, might try

increasing --transfers
add --buffer-size=0

asdffdsa · January 12, 2023, 9:09pm

no bug, that is to be expected.

some app, perhaps media server, such as plex is asking rclone to iterate over the dir/files.
or
ExecStartPost command does that.

600K lines divided by 4 = 250K files. and that might be a lot of files for onedrive, which has lots of throttling.

rasmaplasma · January 13, 2023, 3:44pm

tried with transfer=6 and buffer-size=0, neither seemed to affect the upload speed, is they anything else i could try?

asdffdsa · January 13, 2023, 3:54pm

might try tweaking --onedrive-chunk-size

asdffdsa · January 13, 2023, 9:46pm

perhaps @Ole has a suggestion?

lairai · January 14, 2023, 6:46am

We have nothing to do here. It's OneDrive limit which is the reason people prefer GG Drive than OneDrive. On the same machine, GG Drive upload speed 80-165MB/s, OneDrive is just 4-7.5MB/s.
You might search on Reddit or other forums, they all reported the issue long time ago and no fix until now.

Ole · January 14, 2023, 9:00am

The config looks good to me, I don’t think there is anything we can do to make it faster.

Agree, the speed of OneDrive seems to depend on the subscription plan and price. The more you pay the faster it is.

OneDrive also sometimes slows down the upload of large files, especially if you already uploaded a lot within the past 24 hours.

It looks like an intentional business decision to me.

jared · January 15, 2023, 3:39am

try doing an rclone sync with --transfers=1 flag on the single large file you're talking about. see what the bandwidth is then. as one of the resident onedrive complainers on this forum, you should be able to get 50mbps on both personal and business onedrive's with a single file transfer, assuming you're within a normal latency perimeter of the data center.

jared

rasmaplasma · January 15, 2023, 9:45am

tried that just now, I'm still getting ~5Mi/s, so I guess it's some other issue, maybe I'm just too far away from the data center like you said.

oh well, looks like I'll have to find another way to store my content. But thanks for the help everyone, I really appreciate it.

Ole · January 15, 2023, 10:28am

That is roughly the same as this:

The first is counting bytes transferred by rclone, the second counting bits on the transmission line. The ratio is roughly 10.

jared · January 15, 2023, 2:18pm

thanks @Ole , almost missed that. yeah I think you're getting everything that Microsoft graph gives for single file transfer. I don't think that's too bad actually, for what I do with it I guess if you're dealing single large files and trying to save space on the local disk by using rclone mount, I can see how you'd want more. however, at that point I don't see how double or even quadruple of that would be enough. probably time for a local storage server.

rasmaplasma · January 15, 2023, 2:58pm

Ah yeah you're right, forgot about the different unit there for a sec.

I would, but this is all on a vps haha

thanks for all the help guys

system · February 14, 2023, 2:58pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.