Backup script for Linux

With rclone you can easily make a full backup of your server, thanks a lot to everyone involved. But daily, full backups to the same target mean that if you accidentially delete a file, and first discover this a few days later, the file can't be recovered. Instead, you can make a full, daily backup to a new target each night, but this is slow and takes up a lot of space. Here is a small script I use for rotating, incremental backups.

The script makes a remote copy of selected directories to Amazon S3, in the directory recent . When a file is changed in recent , the previous version is moved to one of a rotating set of remote directories. If you have a set of 7 rotating directories, and you run the script daily, you will be able to go a week back in history and find your missing file there. If you just want the most recent version of your files or directories, you can find them in recent .

The first time the script is run, it will most likely take a few hours. After this, it will be quite fast. 10 minutes for my server.

I will be happy if others can use the script and thankful if someone can help to improve it.


SERVER=$(hostname -s)
VERBOSE=""                  # Add more v's for more verbosity: -v = info, -vv = debug
LOG="/tmp/rclone_log.txt"   # Temporary log file

# bak.list is a file with names of extra backup-versions, starting with the oldest
OLDEST=$(head -1 $BACKUPLIST | sed 's/ .*//')  # the oldest, extra backup-version
COUNT=$(cat $BACKUPLIST | wc -l)               # rotate the file, deleting the first entry and adding a new entry at the end
echo $OLDEST $(date) >> $BACKUPLIST.tmp

# Purge oldest version
echo "Purging oldest version ($OLDEST)..."
nice rclone purge ${REMOTE}/${OLDEST} ${VERBOSE} &>> ${LOG}
echo "...done"

# Perform actual backup
echo "Performing backup to ${REMOTE}..."
nice rclone sync / ${REMOTE}/recent --backup-dir ${REMOTE}/${OLDEST} --filter-from ${FILTER} ${VERBOSE} --skip-links --update --use-server-modtime &>> ${LOG}
echo "...done"
cat ${LOG} >> log.txt

Here is a sample filter:

+ /etc/group/**
+ /etc/gshadow/**
+ /etc/passwd/**
+ /etc/samba/secrets.tdb/**
+ /etc/samba/smbpasswd/**
+ /etc/shadow/**
+ /etc/ssh/**
+ /etc/sudoers/**
+ /home/e-smith/**
+ /opt/search/**
+ /root/**
+ /usr/lib/mailman/aliases/**
+ /usr/share/wordpress/wp-content/**
+ /var/lib/mailman/**
+ /var/log/**
- *

And a sample bak.list file

old5 Thu Aug 6 01:30:01 CEST 2020
old4 Fri Aug 7 01:30:01 CEST 2020
old3 Sat Aug 8 01:30:01 CEST 2020
old2 Sun Aug 9 01:30:01 CEST 2020
old1 Mon Aug 10 01:30:02 CEST 2020
old7 Tue Aug 11 01:30:01 CEST 2020
old6 Wed Aug 12 01:30:01 CEST 2020
1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.