Automating rclone (crontab, etc)

Good afternoon! I currently use rclone to backup many different file sources to different cloud providers. I'm looking at ways to automate this. What is the simplest way, I have a good many I would like to automate?

Thanks in advance for the assistance.

-David

Have you considered using Crontab + Rsync?

For example, running something like:

rsync -aP /path/to/source/ /path/to/destination

running this command again would mean that only the changed files are synced - nothing else!

Just a thought, I am no expert :slight_smile:

I use rync now. I was looking for an easy way to automate the process. Crontab was def on the table. That is actually what I ended up doing. Thanks!

I use it from cron on a nightly basis to send my backups offsite. I just run this script

#!/bin/ksh -p

if [ -t 1 ]
then
  stats=60s
else
  stats=0
fi

rc()
{
  echo
  cd /$1 || exit
  echo $(date) Starting $3

  /home/sweh/bin/rclone -v --stats=$stats --config /home/sweh/.config/rclone/rclone.conf $2 . $3 $4 2>&1 | grep --line-buffered -v symlink
  echo $(date) Ended $3           
}

rc /BACKUP sync OneDriveBackups: '--exclude /old_stuff/**' | grep --line-buffered -v 'INFO.* Copied .new' | grep --line-buffered -v 'INFO.* Deleted$'

rc /Media/mp3 sync OneDriveMusic:mp3
rc /Media/audio sync OneDriveMusic:flac

I don't need to be told about add/deletes to the /BACKUP directory 'cos I expect that to change every night, but if the mp3 or flac area changes I want to know.

@sweh while I write scripts to automate some things, I'm a little lost at the script -- even walking through it slowly.

You have some parameters, got it.

When you are changing directories "cd /$1" what does the $1 represent? I basically am not following the code from that point. I understand the obvious things like config file locations, just a lot more complicated than how I run it.

If you could explain the steps, I would appreciate it.

  • rc() { ... } is a shell function definition with 4 positional parameters
  • rc arg1 arg2 arg3 arg4 is an invocation example where
    • $0 == "rc"
    • $1 == "arg1"
    • $2 == "arg2"
    • $3 == "arg3"
    • $4 == "arg4"
  • @sweh is using KornShell (/bin/ksh)
  • I typically use Bash (/bin/bash)
  • Both are similar with respect to Shell Functions

Bash Reference Manual

rc() is a function. It takes various parameters. These parameters are called positional parameters. The first parameter passed is stored in$1, the second in$2 and so on.

So when I make a call such as

rc /BACKUP sync OneDriveBackups: '--exclude /old_stuff/**' 

then I'm calling the function rc() and $1 $2 etc get populated with the parameters.

In this case

$1 == /BACKUP
$2 == sync
$3 == OneDriveBackups:
$4 == '--exclude /old_stuff/**'

So the cd $1 becomes the same as cd /BACKUP. The same applies to the echo and rclone lines.

hi,

rsync does not copy to cloud storage.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.