Rclone incremental backup

Feature
1

Hi

Hope you all are doing great

Our goal is incremental backup of data directory to storj. we are using following command to backup data incrementally

rclone sync --progress /var/www/html/nextcloud/data/ remote:bucketname/data/

Now it is being observed, when we run above command second time, it deletes the files from storj that were deleted from our data directory (local) & backed-up in first time reclone sync command

We want rclone to keep all files that were previously backed-up and do only incremental backup to storj. so if any file deleted a month or year ago, we can retrieve it from storj regardless how many time we synced our data directory.

looking forward for support

2

Hi Zohaib,

I guess you are looking for the rclone copy command or the --backup-dir flag.

3

Hi thank you for your reply

can you please make a command for my case and bit elaborate its functionality

rclone copy --progress /var/www/html/nextcloud/data/ remote:bucketname/data/

rclone copy compares and copy only modified and new files/directories to destination. Will it keep the older backup files of previous backup if deleted from source before new backup?

how to use --backup-dir and what will be its use?

you are right sync is not appropriate in my case it makes identical source and destination which is not requirement.

4

I want to make it as simple as possible

Is the following command well enough for incremental backup and keep save older backup files that were deleted from source? So we will be able to recover accidentally deleted files after a month or a year regardless how many time rclone copy run for incremental backup

rclone copy --progress /var/www/html/nextcloud/data/ remote:bucketname/data/

Please advice. Thanks

5

You will need to add "--backup-dir remote:bucketname/backups/$(date -Is)" and change the copy to sync.

What this will do is store any files which were deleted or updated in the backups/2022-03-29T11:28:44+01:00 directory where the time shown is the time the command was run.

This assumes you are running Linux - you'll need a different command line for Windows.

6

Hi thank you for your reply.

Is it possible to have deleted files in same directory. is it necessary to use --backup-dir.

I read in documentation rclone sync makes identical source and destination. So i think about rclone copy.

7

hello and welcome to the forum,

I think you want a forever forward incremental backup.

each time this command is run,

rclone sync  /var/www/html/nextcloud/data/ remote:bucketname/data/full --backup-dir=remote:bucketname/data/incrementals/`date +%Y%m%d.%I%M%S` -vv --dry-run

if a file is deleted in the source, then rclone will

  1. create a subdir under incrementals with a date/timestamp.
  2. move the corresponding dest file into that subdir.
8

You can use --suffix but you will need to exclude them from the backup if you are using rclone sync.

If you don't want deletes synced then rclone copy will work fine.

It will make restoring harder though as you won't know which files have been deleted or not.

9

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.