How to have cp, mv, sonarr be as fast as rclone copy

What is the problem you are having with rclone?

I'm using rclone with a jellyfin/sonarr/transmission/sabnzbd setup. I download files locally then move them to a mounted shared google drive.

Here's how I mount the drive:
/usr/bin/rclone mount SDrive0Crypted: /data --allow-other --allow-non-empty --dir-cache-time 240h --log-level INFO --log-file /home/rclonecache.log --poll-interval 15s --umask 002 --user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36" --cache-dir=/cache --vfs-cache-mode full --vfs-cache-max-size 250G --vfs-cache-max-age 240h --bwlimit-file 16M --fast-list --drive-stop-on-upload-limit

When doing a rclone copy, I can reach very high speeds (even when file is not cached). It quickly reaches at least 80Mi/s:

rclone copy --progress SDrive0Crypted:media/movies/bigMovie.mkv /tmp/BIG

Then I try without rclone copy, on another file, doing for exemple:

pv /data/media/movies/anotherBigMovie.mkv > /tmp/BIG2

Now the speed is maxed at 16Mi/s. Maybe because it's not multi-threaded?

I'm guessing sonarr & radarr use something simple like mv to move files, because their operations are very slow also. How to either speed up cp & mv or make radarr/sonarr use rclone copy?

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.2

  • os/version: debian 11.5 (64 bit)
  • os/kernel: 5.10.0-8-amd64 (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.18.6
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google shared enterprise drive

The rclone config contents with secrets removed.

[SDrive0]
type = drive
client_id = REDACTED
client_secret = REDACTED
scope = drive
token = {REDACTED}
team_drive = REDACTED
root_folder_id = REDACTED

[SDrive0Crypted]
type = crypt
remote = SDrive0:
password = XXX
password2 = XXX

Mounts are not multi threaded so you can't compare the two in terms of speed.

There's one issue for upload and one issue for download:

Multi-thread upload for different backends · Issue #4798 · rclone/rclone (github.com)

Issues · rclone/rclone (github.com)

Thanks, so what I get is that it's a current rclone limitation.

Can you guys tell me if you also have average of 15Mi/s with google drive?

I can't as I don't use Google Drive anymore.

My mounts are generally fire and forget and items get uploaded later so I have no concerns if it takes 1 minute or 10 minutes as it's all behind the scenes for me.

You have a bandwidth limit on the mount as well:

Which is probably why you get capped at that level.

I missed that, thanks a lot!!! I put it at 200M
My speed is now about 40Mi/s which is much better.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.