[SOLVED] rClone move script leading to heavy load

Help and Support
1

Hi,

I use rClone 1.36 (not BETA) on a Debian8 CT. Also tested on dedicated servers, the same.
I wanted to test @Ajki script for the upload (thank you BTW!): https://github.com/ajkis/scripts/blob/master/rclone/rclone-upload.sh

Here’s my version:

#!/bin/bash
# RCLONE UPLOAD CRON TAB SCRIPT 
# Type crontab -e and add line below (without #)
# * * * * * /home/plex/scripts/rclone-upload.cron >/dev/null 2>&1

if pidof -o %PPID -x "rclone-upload.cron"; then
   exit 1
fi


LOG="/home/XXX/Logs/rclone-upload.log"
FROM="/home/XXX/Pre/"
TO="ACD_Enc:/"
EXC1="/PreFilms/**"
EXC2="/PreSeries/**"

# CHECK FOR FILES IN FROM FOLDER THAT ARE OLDER THAN 15 MINUTES
if find $FROM* -type f -mmin +20 | read
  then
  echo "$(date "+%d.%m.%Y %T") RCLONE UPLOAD STARTED" | tee -a $LOG

/usr/sbin/rclone move "$FROM" "$TO" -c -v --no-traverse --transfers=16 --checkers=32 --delete-after --exclude="$EXC1" --exclude="$EXC2" --min-age 20m --log-file="$LOG"
  echo "$(date "+%d.%m.%Y %T") RCLONE UPLOAD ENDED" | tee -a $LOG

Everything is fine for 10-15 minutes after CT/server upstart, I got about 10-12 connections to my cloud and then I got more and more processes/connections till the CT/servers load is too heavy I got to stop/reboot it.

Any idea on what’s going on? thank you

6of9aBP.jpg1116×737 276 KB

y8X18X4.jpg1116×737 127 KB

GrIRVF2.jpg1116×737 137 KB

2

Those aren’t processes, but threads in htop which is totally normal.

You are moving 16 files at a time with transfers=16. If that’s too much, make it lower or lower the bandwidth.

I posted the htop info -> How many rclone mount sessions should my server be running?

3

Thanks but I still can’t understand that load: 138 249 180. As this CT is running rClone/script only (rTorrent stopped).

4

Because you are transferring 16 files, make that number lower. Try the default at 4 and work your way up or down.

5

I’ll go for that and get back to you.

6

Got it, the script runs too many times… this part didnt work.

7

That would do it too :slight_smile:

Glad you found that as well.

1 Like