Trying to achieve continuous transfer of files for rclone move, especially rclone picking up newly added files of the source folder. I've been using sonarr&radarr and previously made sonarr to move files to remote (Onedrive) directly but this resulted in very high iowait and thus unnecessary system slowdown.
I have tried using mergerfs to merge local and remote then moving by scheduled cron job, but some of my files are huge and take several hours to transfer and this causes new downloaded files to queue up. For example when a single 4k remux file has started to move, it uses maximum 100mbps and takes about two hours to upload and during that 2 hours other files cannot move and I cannot utilize 1gbps connection
I think this will be solved by starting a rclone move job that picks up new files that gets added to the source folder. Is there any way to achieve this, either by using rclone commands or scripts?
Run the command 'rclone version' and share the full output of the command.
rclone v1.57.0
os/version: ubuntu 20.04 (64 bit)
os/kernel: 5.4.0-100-generic (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.17.2
go/linking: static
go/tags: none
Which cloud storage system are you using? (eg Google Drive)
Microsoft Onedrive
The command you were trying to run (eg rclone copy /tmp remote:tmp)
Paste command here
The rclone config contents with secrets removed.
[disk0]
type = onedrive
client_id =
client_secret =
token =
drive_type = business
I might not have been clear with my problem. I currently use --vfs-cache-mode full mount based on your mount service
ExecStart=/usr/bin/rclone mount disk0: /home/spark/media/ \
--allow-other \
--dir-cache-time 10s \
--log-file /home/spark/logs/rclone.log \
--log-level INFO \
--cache-dir=/home/spark/cache/disk0 \
--vfs-cache-mode full \
--vfs-cache-max-size 30G \
--vfs-cache-max-age 5000h \
The problem is that if I point sonarr/radarr directly to the mount and let them transfer the files, I end up with excessive iowait and system slowdown. If I use your method of using mergerfs, the sonarr importing part goes smoothly but if the scheduled rclone move happens to start when a single 4k remux file is downloaded, it queues up the future downloads and I would like to speed up the process as if it was a mount. (Also when handling large files rclone move seem to be much reliable than mount especially with Onedrive hash mismatch errors)
You didn't include a mount command nor a log file so it isn't that you were not clear, you neglected to post the information so I guessed based on the lack of effort in your first post.
Not sure what that means as there's no logfile.
I've been running with no mergerfs writing to the mount for a few weeks now. No issues at all. So really need a log file.
Reproduce the issue. Share a log file as asked for in the template.