Is there a built-in way to prevent multiple identical sync instances?

What is the problem you are having with rclone?

I will be invoking rclone from a cron job that starts up on a repeating schedule to sync content that changes on a random basis. The run time will vary from a few minutes if there's nothing to do, to many hours for large amounts of data, possibly running past the next scheduled invocation time. I want to prevent another instance from running if the previous instance is still in progress. I know I can write a script to do this, but I was wondering if rclone has any clean built-in way of achieving this?

Run the command 'rclone version' and share the full output of the command.

rclone v1.57.0-DEV
- os/version: redhat 7.9
- os/kernel: 3.10.0-1160.81.1.el7.ppc64le (ppc64le)
- os/type: linux
- os/arch: ppc64le
- go/version: go1.20.2
- go/linking: dynamic
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

sftp

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Just any regular sync command:

rclone sync remote:source destination

The rclone config contents with secrets removed.

n/a

A log from the command with the -vv flag

n/a

hello and welcome to the forum,

the forum has many examples of using cron jobs.
https://forum.rclone.org/search?q=cron

https://forum.rclone.org/t/what-am-i-missing-in-crontab-executing-the-cron-files/10926
the key code is

if pidof -o %PPID -x "$0"; then
** exit 1**
fi

or use flock
Script works but i have some stalling/ major issue
flock -x -w 5 200 || { echo "script is already running"; exit 1; }

If you want a cheap way of preventing multiple rclone's running use the --rc flag. Only one rclone can listen at a port at once. You can set --rc-addr localhost:9999 also to change the port number if you want groups of rclone's that run at once.

Okay, that should definitely be accompanied by comments in the crontab explaining what's going on, but it does sort of achieve the desired goal rather nicely. :upside_down_face: I did expect any "elegant" solution to require a unique identifier of some kind so that conflicting instances could "agree" that they were performing the same task and were thus in conflict, and the port number meets that need quite well. You wouldn't want to prevent other non-conflicting rclone instances from running... Thanks!

Well I did say cheap!

If you are using a recent rclone you can use a socket file for --rc-addr so you could document using --rc-addr /tmp/rclone-backing-up-my-stuff

1 Like

I was going to mention in my previous reply that a socket file might be an even better way to self-document the usage. Thanks.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.