Rclone cache high CPU and memory usage while adding file to cache-tmp-upload-path

I’ve gone back to trying to get rclone cache working as I just can’t get unionfs to work for me (sonarr/radarr won’t import new files). My situation is I’ve created a fresh cache-tmp-upload-path and I’m transferring old pending files from the old cache-tmp-upload-path (different folder) via a standalone rclone move job i.e. the new cache-tmp-upload-path is only getting new files.

At the moment radarr is adding a 17GB file. Not only is it taking forever, but it’s hammering my RAM and CPU:

I have a good spec system - E5-2683V3 and 64GB of ram so this is very surprising. Are there any settings on rclone cache that will stop it being a resource hog or speed it up? e.g. could the pending files go to disk rather than ram if that’s what’s happening? I only use the cache to upload files to gdrive and to keep radarr/sonarr in sync as I play files via a separate vfs mount, so I dont’ need things to happen in ram for ‘speed’

I must be doing something wrong as this can’t be normal. Thanks in advance for any help

# Mount rclone vfs mount

rclone mount --allow-other --dir-cache-time 96h --cache-dir=/tmp/rclone/vfs --vfs-read-chunk-size 64M --vfs-read-chunk-size-limit 1G --buffer-size 256M --syslog --umask 002 --rc --log-level INFO --stats 1m gdrive_media_vfs: /mnt/disks/rclone_vfs &

# Mount rclone cache

rclone mount --allow-other --dir-cache-time=360h --cache-chunk-path=/tmp/rclone/cache --cache-db-path=/tmp/rclone/cache --cache-chunk-size=512M --cache-total-chunk-size=8G --cache-info-age=368h --cache-db-purge --cache-tmp-upload-path=/mnt/user/rclone_upload --cache-tmp-wait-time=1h --log-level INFO --stats 1m gdrive_media: /mnt/disks/rclone_cache &

# Mount old rclone cache so can decrypt files and upload via rclone move job

rclone mount --allow-other --dir-cache-time 5m --cache-dir=/tmp/rclone/old_cache --vfs-cache-max-age 1h --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit 512M --buffer-size 128M --syslog --umask 002 --rc --log-level INFO --stats 1m rclone_cache_old: /mnt/disks/rclone_cache_old &

19% CPU really isn’t high if you are running encryption.

Without seeing what files are being used, high memory usage may or may not be bad as it uses memory based on your buffer size.

Run a 'lsof " as your root user and see what files are being used.

i think I’ve posted many times that cache only uses 1 worker so it’s going to be slow to upload.

Copying to the cache isn’t slow as it comes down to your write speed of you hard disk. You’d see some slowness with copying many things are you are probably getting IO bound.

You can use iotop to see.