ale82x
(Ale82x)
March 12, 2023, 10:11am
1
yes
Which cloud storage system are you using? (eg Google Drive)
google drive
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
rclone copy /share/folder1/ remote:/folder1/current --backup-dir remote:/folder1/`date +%Y-%m-%d__%H-%M-%S`/ -v --log-file=/var/log/rclone.log --bwlimit=2M
The rclone config contents with secrets removed.
[remote
type = drive
client_id = xxxxxxxxxxxxx
client_secret = xxxxxxxxxxxxxxx
scope = drive
service_account_file = /my/json.json
team_drive = xxxxxxxxxxxxx
root_folder_id =
acknowledge_abuse = true
stop_on_upload_limit = true
stop_on_download_limit = true
A log from the command with the -vv
flag
no error , so i can't copy log
with command posted before works great, i would insert in a script and with a single folder work great
my script is
rclone copy /share/folder1/ remote:/folder1/current --backup-dir remote:/folder1/`date +%Y-%m-%d__%H-%M-%S`/ -v --log-file=/var/log/rclone.log --bwlimit=2M
wget --no-check-certificate https://hc-ping.com/<id>/$?
exit
so i copy my files from nas or linux and put exit code on https://healthchecks.io
so i'm sure that "backup" that work with exit code 0
now i would extend this script for my friend on his NAS.
he have more folder so i think script becomes
rclone copy /share/folder1/ remote:/folder1/current --backup-dir remote:/folder1/`date +%Y-%m-%d__%H-%M-%S`/ -v --log-file=/var/log/rclone.log --bwlimit=2M
rclone copy /share/folder2/ remote:/folder2/current --backup-dir remote:/folder2/`date +%Y-%m-%d__%H-%M-%S`/ -v --log-file=/var/log/rclone.log --bwlimit=2M
wget --no-check-certificate https://hc-ping.com/<id>/$?
exit
and other folder (in total are 10 shared folders)
my problem is exit code, how can i have only 1 exit code to send/check with https://healthchecks.io ?
thank you in advance
asdffdsa
(jojothehumanmonkey)
March 12, 2023, 1:37pm
2
try something like
create a text file with the name of the folders, list.txt
/folder1/**
/folder2/**
have rclone use the list, only those two folders will be copied.
rclone copy /share remote:current --include-from=list.txt --backup-dir=remote:folders/`date +%Y-%m-%d__%H-%M-%S`/ -v --log-file=/var/log/rclone.log --bwlimit=2M
ale82x
(Ale82x)
March 12, 2023, 7:38pm
3
thank you.
i try to use with list.txt
but when i run my script i have exit code 0 (so seems ok)
but in log file there is
2023/03/12 20:35:00 INFO : Starting bandwidth limiter at 2Mi Byte/s
2023/03/12 20:35:03 INFO : There was nothing to transfer
2023/03/12 20:35:03 INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 3.0s
but name of different folder is correct...
thank you
asdffdsa
(jojothehumanmonkey)
March 12, 2023, 7:42pm
4
ok, run the command with -vv
ale82x
(Ale82x)
March 12, 2023, 9:26pm
5
sorry but I hadn't thought of that
here log with -vv
2023/03/12 22:19:22 INFO : Starting bandwidth limiter at 2Mi Byte/s
2023/03/12 22:19:22 DEBUG : rclone: Version "v1.61.1" starting with parameters ["/share/homes/admin/rclone" "copy" "/share" "remote:/bck-rclone2/" "--include-from=/share/homes/admin/list.txt" "--backup-dir" "remote:/bck-rclone2/folders/2023-03-12__22-19-22/" "-vv" "--log-file=/share/homes/admin/script_bck_rclone2.log" "--bwlimit=2M"]
2023/03/12 22:19:22 DEBUG : Creating backend with remote "/share"
2023/03/12 22:19:22 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/03/12 22:19:22 DEBUG : Creating backend with remote "remote:/bck-rclone2/"
2023/03/12 22:19:27 DEBUG : fs cache: renaming cache item "remote:/bck-rclone2/" to be canonical "remote:bck-rclone2"
2023/03/12 22:19:27 DEBUG : Creating backend with remote "remote:/bck-rclone2/folders/2023-03-12__22-19-22/"
2023/03/12 22:19:31 DEBUG : fs cache: renaming cache item "remote:/bck-rclone2/folders/2023-03-12__22-19-22/" to be canonical "remote:bck-rclone2/folders/2023-03-12__22-19-22"
2023/03/12 22:19:31 DEBUG : .proftpd.fifo: Excluded
2023/03/12 22:19:31 DEBUG : CACHEDEV1_DATA: Excluded
2023/03/12 22:19:31 DEBUG : external: Excluded
2023/03/12 22:19:31 DEBUG : HDZ_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDY_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDX_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDW_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDV_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDU_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDT_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDS_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDR_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDQ_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDP_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDO_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDN_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDM_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDL_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDK_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDJ_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDI_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDH_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDG_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDF_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDE_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDD_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDC_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDB_DATA: Excluded
2023/03/12 22:19:31 DEBUG : HDA_DATA: Excluded
2023/03/12 22:19:32 DEBUG : Google drive root 'bck-rclone2': Waiting for checks to finish
2023/03/12 22:19:32 DEBUG : Google drive root 'bck-rclone2': Waiting for transfers to finish
2023/03/12 22:19:32 INFO : There was nothing to transfer
2023/03/12 22:19:32 INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 9.9s
2023/03/12 22:19:32 DEBUG : 12 go routines active
of course that excluded is name of folder that i DO NOT WANT backup, so seems ok but "was nothing to transfer"
thank you again
asdffdsa
(jojothehumanmonkey)
March 12, 2023, 9:32pm
6
not sure,
tho, maybe
remote:/bck-rclone2
should be
remote:/bck-rclone2/current
ale82x
(Ale82x)
March 13, 2023, 6:06am
7
same...
2023/03/13 06:57:49 DEBUG : rclone: Version "v1.61.1" starting with parameters ["/share/homes/admin/rclone" "copy" "/share" "remote:/bck-rclone2/current/" "--include-from=/share/homes/admin/list.txt" "--backup-dir" "remote:/bck-rclone2/folders/2023-03-13__06-57-48/" "-vv" "--log-file=/share/homes/admin/script_bck_rclone2.log" "--bwlimit=2M"]
2023/03/13 06:57:49 DEBUG : Creating backend with remote "/share"
2023/03/13 06:57:49 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/03/13 06:57:49 DEBUG : Creating backend with remote "remote:/bck-rclone2/current/"
2023/03/13 06:57:50 DEBUG : fs cache: renaming cache item "remote:/bck-rclone2/current/" to be canonical "remote:bck-rclone2/current"
2023/03/13 06:57:50 DEBUG : Creating backend with remote "remote:/bck-rclone2/folders/2023-03-13__06-57-48/"
2023/03/13 06:57:51 DEBUG : fs cache: renaming cache item "remote:/bck-rclone2/folders/2023-03-13__06-57-48/" to be canonical "remote:bck-rclone2/folders/2023-03-13__06-57-48"
2023/03/13 06:57:51 DEBUG : .proftpd.fifo: Excluded
2023/03/13 06:57:51 DEBUG : CACHEDEV1_DATA: Excluded
2023/03/13 06:57:51 DEBUG : external: Excluded
2023/03/13 06:57:51 DEBUG : HDZ_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDY_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDX_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDW_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDV_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDU_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDT_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDS_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDR_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDQ_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDP_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDO_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDN_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDM_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDL_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDK_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDJ_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDI_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDH_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDG_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDF_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDE_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDD_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDC_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDB_DATA: Excluded
2023/03/13 06:57:51 DEBUG : HDA_DATA: Excluded
2023/03/13 06:57:51 DEBUG : Google drive root 'bck-rclone2/current': Waiting for checks to finish
2023/03/13 06:57:51 DEBUG : Google drive root 'bck-rclone2/current': Waiting for transfers to finish
2023/03/13 06:57:51 INFO : There was nothing to transfer
2023/03/13 06:57:51 INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 2.7s
if i use this script
#!/bin/bash
REMOTE=remote:/bck-rclone
DEST_DATE=`date +%Y-%m-%d__%H-%M-%S`/
LOG=/share/homes/admin/script_bck_rclone.log
BWLIMIT=2M
FOLDER1=folder1
/share/homes/admin/rclone copy /share/$FOLDER1/ $REMOTE/$FOLDER1/current/ --backup-dir $REMOTE/$FOLDER1/$DEST_DATE -v --log-file=$LOG --bwlimit=$BWLIMIT
if [[ $? -ne 0 ]]; then
FOLDER2=folder2
/share/homes/admin/rclone copy /share/$FOLDER2/ $REMOTE/$FOLDER2/current/ --backup-dir $REMOTE/$FOLDER2/$DEST_DATE -v --log-file=$LOG --bwlimit=$BWLIMIT
if [[ $? -ne 0 ]]; then
wget --no-check-certificate https://hc-ping.com/<id>/$? -T 10 -t 5 -O /dev/null
fi
fi
exit
so if i have exit code 0 on first "copy" switch to second (of course indeed i use ten if)
and when the last finish with exit code 0 send to healthcheck
is not clean but seems my limit ...
thank you