Can we use Rclone as incremental backup tool?

Help and Support
1

Hi, I have installed Rclone today on my server and configured it perfectly with Wasabi, now I want to run rclone once per day for taking an incremental backup of my files.

I have around 1.1 TB files and increasing daily so I am thinking first I will copy all files and then sync once per day only new files, old files remain the same.

Can anyone please guide me on how can I do it?

2

You can, but Rclone is not designed to be used as an incremental backup program. If you want to run incremental backups of your data, you really should be using a program designed for that. I use kopia for my incremental backups and there are others too like restic and duplicati

1 Like
3

hello and welcome to the forum,

for many years, on a daily basis,
i use rclone as an incremental file backup tool, using wasabi.

rclone sync only copies new/changed files, so in a sense that is incremental.
tho the question is, what to do with an old file in wasabi, that rclone will overwrite with a newer file from local

  1. compare local file to wasabi file.
  2. determined that the local file is newer than the older file in wasabi.
  3. need to decide what to do with that older file in wasabi
    --- a. have rclone sync delete the older file in wasabi and replace it with newer local file.
    --- b. have wasabi save the older file as a version.
    --- c. use a command like so, which is the equivalent to forever forward incremental backup.
rclone sync /path/to/source remote:full --backup-dir=remote:incrementals/`date +%Y%m%d.%I%M%S`

depending on your use-case, might want to use a block-based backup.
on windows, i use veeam backup and replication.

rclone has good support for restic
https://rclone.org/commands/rclone_serve_restic/

1 Like
4

Hi Thank you, so basically sync will copy all files every time?

5

rclone sync will check all files and then

  • copy files that are new or have changed
  • delete files that are deleted

The command can optionally move the the files to be updated/deleted to a history folder with --backup-dir.

The command is highly configurable as described in the docs:
https://rclone.org/commands/rclone_sync/
https://rclone.org/docs/#backup-dir-dir

We need to know a little more about the data you intend to backup to give you better advice. Is it a full system with live databases, a development environment with a lot of small files or just some relatively static user data (documents, photos and a few movies)? It makes a huge difference and rclone isn't always the best choice.

@asdffdsa Feel free to take over when the day starts in your time zone. You are the Wasabi expert!

6

I have been using hashbackup successfully with rclone for many years now and am very happy with it. At the time I investigated it it was the only solution (I was able to find) to do eternal incremental backups. All other tools at some point insisted on doing a full backup once in a while.

7

rclone can do forever forward incremental backups, i use this technique on a daily basis.
for example,

rclone sync --dry-run /home/username/source remote:latest--backup-dir=remote:incrementals/`date +%Y%m%d.%I%M%S` -vv
8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.