What is the problem you are having with rclone?
I want to know the best settings for copying from one rotational HDD to another rotational HDD without fragmenting the files, or the directories.
I reorganized my local Projects folder (terabytes of code (cough
node_modules cough), video, and other assets) and now I'm trying to get it back in sync on the backup as efficiently as possible. I've mirrored some of the changes by hand, but the bulk of the work is sadly a matter of deleting existing files and writing new copies.
At first I was using
rsync, but it was copying large files at 20/mbps on a drive that gets ~150/mbps reads and writes. That seemed odd, so I tested with
cp and saw much faster transfers.
rclone seems to also copy large files closer to the expected speed.
rclone appears to be excessively fragmenting the disk, switching between various files and folders, rather than writing like files at the same time.
2021-02-02 16:01:16 INFO : Sites/webinstall.dev/home/aj/..../zcallback_windows_arm.s: Copied (new) 2021-02-02 16:01:16 INFO : Sites/telebit.io/var/.../blkio.throttle.write_bps_device: Copied (new) 2021-02-02 16:01:16 INFO : Sites/coolaj86.com/root/.../cpu.a: Copied (new) 2021-02-02 16:01:16 INFO : github.com/coolaj86/.../comp.h: Copied (new)
The copy order appears to be random despite using
--order-by name and
What is your rclone version (output from
rclone version rclone v1.53.4 - os/arch: darwin/amd64 - go version: go1.15.6
Which OS you are using and how many bits (eg Windows 7, 64 bit)
macOS Catalina 64-bit
Which cloud storage system are you using? (eg Google Drive)
The command you were trying to run (eg
rclone copy /tmp remote:tmp)
rclone sync -vP \ --transfers=1 --order-by name \ --exclude 'node_modules/**' --exclude '.Spotlight-*/**' --exclude '.cache*/**' \ ~/Projects/ /Volumes/Backup/Projects/
The rclone config contents with secrets removed.
A log from the command with the