Need some script help with incron and waiting to retry

Help and Support
1

Hi
I’ve been setting up a cron job for moving files using rclone, and have been using the script below.
Now I’ve discovered incron (for triggering scripts when new folder are dropped ready for uploading) and would like some help adapting this script.

Some times a second new folder is created while the first is triggering its upload, the file lock works well to stop duplicate uploads, but I would really like it to just wait, retry checking if the file lock has cleared, then run, in order to upload the second folder created.

Any suggestions?
Thanks

#!/bin/bash
# last part should be the path the script is stored in
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/docker/rclone/

# custom file details
LOCKFILE="/tmp/rclone-upload.lock"
LOGFILE="/docker/rclone/rclone.log"
FROM="/data/upload/movies"
TO="primary:/Movies"

# core script
(
if flock -n 9 ; then
        echo ">>> Starting RCLONE upload script: " `date` >> $LOGFILE
        rclone -v --log-file=$LOGFILE move $FROM $TO
        echo "<<< RCLONE upload script has finished: " `date` >> $LOGFILE
else
        echo "***************************************************************" >> $LOGFILE
        echo "*** ...rclone upload script already running... exiting now. ***" >> $LOGFILE
        echo "***************************************************************" >> $LOGFILE
        echo "" >> $LOGFILE
        exit 1;
fi
) 9>$LOCKFILE
2

You could tell the script to wait while lock file exist.

while [ -e /tmp/rclone-upload.lock ];
do
sleep 5
done

3

I tried the script, so it looks like this…

#!/bin/bash
# last part should be the path the script is stored in
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/docker/rclone/

# custom file details
LOCKFILE="/tmp/rclone-upload.lock"
LOGFILE="/docker/rclone/rclone.log"
FROM="/data/upload/movies"
TO="primary:/Movies"

# core script
(
if flock -n 9 ; then
        echo ">>> Starting RCLONE upload script: " `date` >> $LOGFILE
        rclone -v --log-file=$LOGFILE move $FROM $TO
        echo "<<< RCLONE upload script has finished: " `date` >> $LOGFILE
else
        while [ -e $LOCKFILE ];
        do
        sleep 1
        done
) 9>$LOCKFILE

but it comes back with this error…

./monitor.sh: line 22: syntax error near unexpected token `)'
./monitor.sh: line 22: `) 9>$LOCKFILE'

Any ideas?
Thanks

4

You need to close your if loop.

Add fi On second last line

5

Thanks yes that seems to have fixed it.

6

Iam not really sure this is the best way to to do it, especially since you are just sending it to ELSE statement to wait until lock file is deleting and then it will exit eg not run rclone move command.

Also its a bit counterproductive as well since you will run multiple itteration on every new file eg copy 10 new videos to upload folder (with few seconds delay for each) would run 10 iteration of script, while only 1 is running and other 9 would wait for first one to finish, then 2nd one would actually upload all files found in upload and other 8 are just wasted as there is nothing more to upload. ( assuming you are not adding new movies during that time )

Iam running a crontab script every minute that will automatically exit if upload is already in progress or there are no files for upload.

645e21a94e.png1035×374 26.3 KB

Source: https://github.com/ajkis/scripts/blob/master/rclone/rclone-upload.cron

p.s. Will eventually upload to github all scripts Iam using atm