Rclone sync remote to remote disk usage

I adjusted a sh script from /u/jiru443 (link) and modified it with my own rclone parameters.

I'm going to use it on a VPS where the source is my NAS at home (SFTP) and the destination is my gsuite drive. (I know that I can directly run the script from my NAS and upload directly to gsuite but I want to give this method a try)

Could someone explain me how much disk size I actually need on the vps to do an rclone sync in this scenario? Imagine syncing a 20GB file from my local NAS to the gsuite drive, will rclone first 'download' it to the VPS and then 'upload' it to gsuite. Or will rclone 'stream' the file directly to gsuite?

script i'm using

#!/bin/bash
#variables
LOG="/home/user/rclone/$(date +%F)-rclone.log"
#$(date +%F_%T) to include date + time
#$(date +%F) to include date
LOGLEVEL=INFO
SOURCE="/home/user/downloads/TEST/dir1/"
DEST="gsuite-C:current"
RCLONECONFIG="/home/user/.config/rclone/rclone.conf"
BACKUPDIR="gsuite-C:old"

if pidof -o %PPID -x $(basename $0); then
echo $(date "+%Y/%m/%d %H:%M:%S")" WARN : Cron attempted to start the rclone backup but an existing cron job is still running." >> $LOG
exit 1
fi

echo $(date "+%Y/%m/%d %H:%M:%S")" INFO : Cron started the rclone backup." >> $LOG
echo $(date "+%Y/%m/%d %H:%M:%S")" INFO : Starting sync" >> $LOG

/usr/bin/rclone sync $SOURCE $DEST \
--log-level=$LOGLEVEL \
--tpslimit=10 \
--fast-list \
--config=$RCLONECONFIG \
--checkers=6 \
--transfers=6 \
--max-transfer=200G \
--log-file=$LOG \
--bwlimit=2M \
--retries=1 \
--stats-one-line \
--drive-chunk-size=64M \
--backup-dir=$BACKUPDIR --suffix "_$(date +%FT%T%z).bak" \
--drive-use-trash=true

#--progress --stats-file-name-length=0 --stats=1s \
#uncomment and move upwards to view current progress
#--checksum \
#checksum is not used in crypted remote
#--drive-server-side-across-configs=true \
#add if syncing server side

echo $(date "+%Y/%m/%d %H:%M:%S")" INFO : sync complete" >> $LOG

echo $(date "+%Y/%m/%d %H:%M:%S")" INFO : Cron finished rclone backup." >> $LOG

exit

Please criticize this script as I want to learn as much as I can on shell scripting and using rclone.

You shouldn't need any disk space for buffering files - rclone will stream the file

You script looks like it should work :slight_smile:

Thanks for your answer, I really appreciate it.
And thanks again for the amazing software you created.

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.