Long backups and concurrency

[bobbybuildy (Michael)]

How do you handle running multiple rclone sync processes for that take longer than next cron cycle.

For example, the initial seeding might take 5-15 days, and I might set a cron to run hourly. How do I have it stop if it is running. Do I make a pid file and check that, or is there a better way?

Anyone have an example script?

Also, I am backing up my NAS and I might have more than one rclone sync as they might not be in the same folder root. Is there a way to manage this issue as well as I don’t want two pounding at once.

I could in theory have them all in the same script and the next one wouldn’t run if the first is still going, but does anyone have some examples of what they are doing to solve these two problems?

I am also concerned about being notified if the process doesn’t run properly or doesn’t complete.

[ncw (Nick Craig-Wood)]

Do I make a pid file and check that, or is there a better way?

You can do that. A cheap alternative is to add the --rc flag which will open a local socket for use of the API. You can only have one rclone listening to the socket at once.

I am also concerned about being notified if the process doesn’t run properly or doesn’t complete.

Check out cronic or one of the alternatives to email you on failure of the script.