Nick giving a talk about Rclone and Backblaze

I have actually put a lot of thought into this since I wrote a few sync tools that wrap rclone and/or rsync.

The thing to remember is that rsync is actually two things:

  1. A utility to mirror/one-way-sync one computer to another (but they must be local on one side and ssh (sftp) or local on the other)
  2. An algorithm to propagate small changes of a large file efficiently.

Number 2 is actually a HUGE benefit to rsync. If you have a 10gb file and insert a few bytes at an arbitrary location, rsync will very efficiently make that change. It does this based on rolling checksums and is really clever. It is worth noting that if two files have the same name but completely different content, the algorithm will end up with the correct transfer but slightly less efficient as it is trying to be efficient with the wrong data.

rclone is just number one. It is a tool to mirror (one way again) a set of files and directories from one side to the other. The big difference is that while rsync can only handle ssh (sftp), rclone handles basically everything under the sun. That's where the benefit comes from. And it is a HUGE benefit. As for things like number 2, rclone will repush the entire file.

rclone also has, what I would argue to be a more reasonable API. First, it always does the equivalent of the rsync -a flag. And it skips the nonsense about trailing / that rsync is crazy about (it's not bad once you understand it but it takes some time). However, in addition to the many storage options, it offers some functions rsync doesn't such as rename tracking.

rclone is also SO MUCH MORE than just sync. It has encryption, mounting, serving, interfacing, etc. Some of those are pretty major and useful features.

rsync can also do some tricks that rclone can't. Part of that is because it knows it is only working on real file systems (and often POSIX ones). So it can do things with hardlinks including hardlinking unchanged files and also recreating them on the destination.

So what is the answer to your question:

  • If you will only ever be syncing local to local or SSH and local and you want to efficiently transfer small changes to large files, use rsync.
  • If you want to sync with things other than SSH, you really have no choice but to use rclone. Also while rsync over SSH is encrypted in transit, rclone offers at-rest encryption.
    • Even if it's just SSH, there is benefit to using rclone as you learn the tool that can do it all.
    • (edited to add) rclone can also multi-thread transfers while rsync is serial. I’ve gotten better speeds on local <—> local transfers than rsync because it can do more than one at a time
4 Likes