What is the problem you are having with rclone?
Poor perfomance of / (system) backup on ubuntu 20.04 with an S3 compatible storage object from the same VPS supplier
Run the command 'rclone version' and share the full output of the command.
:~# rclone version -vv
2022/06/03 19:07:22 DEBUG : rclone: Version "v1.58.1" starting with parameters ["rclone" "version" "-vv"]
rclone v1.58.1
-
os/version: ubuntu 20.04 (64 bit)
-
os/kernel: 5.4.0-113-generic (x86_64)
-
os/type: linux
-
os/arch: amd64
-
go/version: go1.17.9
-
go/linking: static
-
go/tags: none
2022/06/03 19:07:22 DEBUG : rclone: Version "v1.58.1" finishing with parameters ["rclone" "version" "-vv"]
Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads
--> YES
Which cloud storage system are you using? (eg Google Drive)
S3 compatible storage
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
/usr/bin/rclone copy --update --transfers 30 --checkers 8 --contimeout 60s --timeout 300s --retries 3 --low-level-retries 10 "/" "mybucket/backup-${today}"
The rclone config contents with secrets removed.
# cat .config/rclone/rclone.conf
[A1]
type = s3
provider = Ceph
access_key_id = XXXXXXXXXXXXXxx
secret_access_key = YYYYYYYYYYYYYY
endpoint = https://ZZZZZZZZZ.com
#
A log from the command with the -vv
flag
# pidof rclone
123456
# ps -p 123456 -o etime
ELAPSED
18:00:20
#
I set the rclone command backup yesterday late at night around 1H00 am and was hoping that it will complete the backup in less then 1 hour! but it took 18H00 and it's still running (but will stop it and kill the rclpone process as it's consuming around 80% of my CPU power!!
I tested rclone before lunching a full backup with a small file of 50MB and it completed the clone in 8seconds!
My system has 17GB in total (around 5GB are just mysql binlog files that started to increase with the latest mysql V 8).
(Bearing in mind the 50MB file backed up by rclone in 8s, the 17000 MB data should be backed up in around 45 min!)
I just needed to have a clean backup before purging those binlog files but rclone is NOT helping at the moment!
# df
Filesystem 1K-blocks Used Available Use% Mounted on
udev 4044892 0 4044892 0% /dev
tmpfs 814820 50944 763876 7% /run
/dev/sda2 49225364 17099592 29592860 37% /
NB: I checked my syslog file, but there's No Data in it about rclone!
(aside the information when the rclone command was launched.)
Appreciate your recommendations on how to fix my rclone parameters and have my backup done in a proper manner.
Thank you!