Backup to Google Drive and delete oldest archive

Help and Support
1

My goal is to back up specified folders as a tar.bz2 archive, upload them to my Google Drive, then delete the archive on a schedule. Idealy, I would have 2 backed up files; 1 that is uploaded to the Google drive as the current backup and another new one, so there are always 2 backup files on my local system, but one file that is uploaded to Google Drive. The oldest backup should be deleted when the newest is created. I’d love to create a script for this, but my knowledge is limited to what I’ve done so far, which is creating 3 cronjobs:

Create the backup

`0 */6 * * * /bin/tar -c --exclude-ignore=/home/bots/.tarignore -vjf bots.tar.bz2 /home/bots/`

Upload to Google Drive

`0 */7 * * * /usr/bin/rclone copy /home/bots/bots.tar.bz2 drive:apps/discord/backups/`

Delete the backup

`0 */8 * * * rm /home/bots/bots.tar.bz2`

Any help creating a script would be extremely helpful.

2

Looks like you’ve done most of the work for your first iteration.

Make a file called backup.sh and in it put

#!/bin/bash
/bin/tar -c --exclude-ignore=/home/bots/.tarignore -vjf bots.tar.bz2 /home/bots/
/usr/bin/rclone copy /home/bots/bots.tar.bz2 drive:apps/discord/backups/
rm /home/bots/bots.tar.bz2

Run chmod a+x backup.sh and use the path to that script in the crontab.

That will make changing stuff easier.

Now for the 3 backups, you could do something like this

#!/bin/bash
/bin/tar -c --exclude-ignore=/home/bots/.tarignore -vjf bots.tar.bz2 /home/bots/
/usr/bin/rclone move drive:apps/discord/backups/bots.tar.bz2.1 drive:apps/discord/backups/bots.tar.bz2.2
/usr/bin/rclone move drive:apps/discord/backups/bots.tar.bz2 drive:apps/discord/backups/bots.tar.bz2.1
/usr/bin/rclone copy /home/bots/bots.tar.bz2 drive:apps/discord/backups/
rm /home/bots/bots.tar.bz2
3

thanks! this waits for each step to complete successfully before going to the next?
also, does rclone only replace the current backup if its 100% uploaded?
i ask because im afraid if a disaster happens and my server dies mid upload, i will have no backup and a corrupt halfway uploaded file

4

also weirdly enough, when i tar normally without the script, it uses the .tarignore i’ve set but the script ignores the .tarignore in /home/bots/.tarignore

5

If you want that then start the script

#!/bin/bash
set -e

Which will cause the script to exit if any of the actions fail. Note that the moves will fail unless there is a file to move, so you might want to create a dummy one first!

That depends on the backend, but with drive that is the case.

It might depend on where you start the tar command so you might want to insert a cd /somewhere in before the tar command.

6

thank you!
here’s the error i get
2018/10/02 22:47:53 ERROR : Google drive root ‘apps/discord/backups/bots.tar.bz2.1’: Server side directory move failed: directory not found
2018/10/02 22:47:53 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
2018/10/02 22:47:53 ERROR : Google drive root ‘apps/discord/backups/bots.tar.bz2.1’: Server side directory move failed: directory not found
2018/10/02 22:47:53 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
2018/10/02 22:47:53 ERROR : Google drive root ‘apps/discord/backups/bots.tar.bz2.1’: Server side directory move failed: directory not found
2018/10/02 22:47:53 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
2018/10/02 22:47:53 Failed to move: directory not found

43%20PM
Screen Shot 2018-10-02 at 6.57.43 PM.png836×440 21 KB

7

What does rclone ls drive:apps/discord/backups print?

8

i think it fixed itself after the second run weirdly enough

9

It would do as that is when all the old versions have been created.

I can see from your screenshot I made a mistake with the move commands

These should be moveto commands not move otherwise it will move them into directories with those names. You’ll want to move the bots.tar.bz2.2 and bots.tar.bz2.1 out the way first.

10

ok so i’ve got this
#!/bin/bash
cd /home/bots
/bin/tar -c --exclude-ignore=/home/bots/.tarignore -vjf bots.tar.bz2 /home/bots/
/usr/bin/rclone move drive:apps/discord/backups/bots.tar.bz2.1 drive:apps/discord/backups/bots.tar.bz2.2
/usr/bin/rclone move drive:apps/discord/backups/bots.tar.bz2 drive:apps/discord/backups/bots.tar.bz2.1
/usr/bin/rclone copy /home/bots/bots.tar.bz2 drive:apps/discord/backups/
rm /home/bots/bots.tar.bz2

11

Change the move commands to moveto, otherwise it will create directories.