Rclone move cronjob repetition

Help and Support
1

I like to keep a folder on my machine called ‘The Drop,’ where I can just drop files in, and they are moved to Google Drive into a folder there for organizing later.

I’d like to always keep all files dropped into there moved over. Would it be a bad idea to run a cron job every 5 minutes to rclone move everything in that directory? Would that cause problems if I had a huge number of files in there and they were still trying to move when the next job initiated?

Thanks.

2

In general, you don’t want to run a job when another one is already running.

I use a pretty simple lock check and that ensures that if a job is still running, it waits for the other to finish so you aren’t hitting a race when trying to upload things.

felix@gemini:~$ cat /home/felix/scripts/upload_cloud
#!/bin/bash
# RClone Config file
RCLONE_CONFIG=/data/rclone/rclone.conf
export RCLONE_CONFIG
LOCKFILE="/var/lock/`basename $0`"

(
  # Wait for lock for 5 seconds
  flock -x -w 5 200 || exit 1

# Move older local files to the cloud
/usr/bin/rclone move /data/local/ gcrypt: --checkers 3 --log-file /home/felix/logs/upload.log -v --tpslimit 3 --transfers 3 --drive-chunk-size 32M --exclude-from /home/felix/scripts/excludes --delete-empty-src-dirs

) 200> ${LOCKFILE}
1 Like