Backup Use Case

Help and Support
1

What is the problem you are having with rclone?

I want to use rclone as more of a backup solution, can i use the "copyto" command to copy all fields from directory A to cloud storage. Then every 3 months or so run "sync" to check directory A against cloud storage and remove anything from cloud storage that is not present in the local filesystem?

Run the command 'rclone version' and share the full output of the command.

rclone v1.61.1

  • os/version: slackware 15.0+ (64 bit)
  • os/kernel: 5.19.17-Unraid (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.19.4
  • go/linking: static
  • go/tags: none

Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads
-->

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

I currently run "sync" on a monthly basis but rather run "copyto" on a daily basis and perform a "sync" every few months to make source and destination match

The rclone config contents with secrets removed.

[GDrive]
type = drive
scope = drive.file
token =
team_drive = 

[secure]
type = crypt
remote = GDrive:CloudBackup
password = 
password2 =

A log from the command with the -vv flag

I am still very dumb and this for some reason is beyond me.
2

yes, but why not just use rclone sync for everything, daily and monthly?

3

Well sync checks source and destination and makes sure destination matches source? Which is what i want BUT..... If a file is deleted on accident wont sync catch that and remove it from destination? IM trying to make it so it copys files over but doesn't remove ANYTHING until i am sure there are no mistakes.

I know what i am trying to say but not sure if im making sense with my explanation. Basically I only want to "prune" the destination every few months when i know my local filesystem is accurate.

4

you might to read the docs about --backup-dir and use a command such as this.
each time rclone sync is run, any files that would have been deleted from secure:full are instead moved to a subdir of secure:incrementals with a date/timestamp. a kind of file versioning.

rclone sync source secure:full --backup-dir=secure:incrementals/`date +%Y%m%d.%I%M%S` -vv
5

so here is my simple command i run now

rclone sync -vv '/mnt/user/Mediastorage/Data1' 'secure:data1' --drive-stop-on-upload-limit

But if i change it do this (it will move deleted files into secure:incrementals?)

rclone sync /mnt/user/Mediastorage/Data1 secure:data1 --backup-dir=secure:incrementals/ date +%Y%m%d.%I%M%S --drive-stop-on-upload-limit -vv

I am also looking through the docs but i dont parse that information as well as when i just ask a straight forward question lol

6

i suggest, create a new local dir and do testing on that

  1. mkdir /mnt/user/Mediastorage/test
  2. touch /mnt/user/Mediastorage/test/file.ext
rclone sync /mnt/user/Mediastorage/test secure:test --backup-dir=secure:incrementals/`date +%Y%m%d.%I%M%S` -vv
  1. repeat step 2
  2. repeat step 3
  3. rclone ls secure:incrementals
7

Well damn, it worked!

Ok last question i swear, if i want to clean out the incremental folder every 6 months is there an rclone command i want to run?

I assume it would be this rclone cleanup

But says not supported by all remotes but doesn't say which ones.

8

it must be --magic :wink:

might try
rclone delete secure:incrementals --min-age=6M --rmdirs --dry-run -vv

rclone cleanup works with gdrive
https://rclone.org/overview/#optional-features

9

2023/02/22 14:05:25 DEBUG : 20230222.125712/Test Delete.txt: Excluded from sync (and deletion)
2023/02/22 14:05:25 DEBUG : 20230222.125712/Test Delete.txt: Excluded
2023/02/22 14:05:25 NOTICE: 20230222.125712: Skipped remove directory as --dry-run is set
2023/02/22 14:05:25 NOTICE: 20230222.125610: Skipped remove directory as --dry-run is set

This is what i get, which looks ok at first glance, it didn't remove the files because they are not older then 6 months, but, in the future when i run this some files will be older then 6 months and some will not be so the rmdir part is a bit concerning, if it removes the directly isn't it essentially deleting everything? Because it was a dry run it left the directory but the logs seem to suggest it would have yeeted that all out of there?

10

as a test, use rclone touch to create a set of files, each with a different modtime.
run your command(s) against that.
tho, i would use test using rclone ls, as that will list files, not delete them.

11

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.