Rclone sync entire drive

Hi rclone team,

A very simple question.

Is it safe to rclone sync an entire 4To drive daily ?

Changes aren't so big, somewhere between 20MB to 50GB daily.
But maybe that would be stressfull for the drives ?
Or that another solution might be more appropriate ?

Best regards. :slight_smile:

it would not be stressful, transferring such a small amount of data daily.

are you trying to rclone for backup of a local drive?
what provider are you using?
what is your operating system?
have you seen this flag?
https://rclone.org/docs/#backup-dir-dir

Hi
Thanks for your answer.

The goal is indeed to have a backup of the source drive.
Both drives are local (internal sata), on a home server running Ubuntu18LTS.
Source drive is a Nextcloud data drive. Both 4To drives identical.
rclone sync is run through a crontab script daily from and to the root of both drives.
Thanks for the tip on the flag. Not sure its appropriate to my use case though.
Although the next step will be to do the same on a 3rd remote drive of the very cold deep storage type like Glacier for catastrophic emergency retrieve only (still studying the cost of that, with daily sync).

Anyhow.
I was worried about the way [which I obviously don't understand] in which rclone sync knows which files are new or changed. I imagine it has to scan the entire drive (even if there are no changes). I don't see how that can't be stressful for the drives ?

Best regards,

Rclone will scan the entire drive. By default it will check size+modification date on the local disk vs size+modification date on the remote and only transfer if those differ.

A scan of the disk isn't particularly intensive - it really depends on how many files you have. It only reads the directory info on the files, not the files (unless you use the --checksum flag).

If you want to simulate it, then do rclone size /path/to/disk this will scan all the files in a very similar way.

What kind of disks are they? SSD or HDD?

Hi

Thanks for your feedback.

I am not planning on using the checksum option at this point in time (although that would be safer for the data I suppose, I most of all don't want to kill my drives too fast).
I'm using SMART monitoring to check on them, and am reasonnably hoping they can last about 2 years.

Both drives (source and backup) are 4TB WesternDigital Red WD40EFRX 3,5"
However, the number of file counts to 239'987 files in 23'066 directories for 2,9TB used.

Again thanks for your feedback, and the beautiful way you're maintaining this great product that is rclone.

Best regards,

I don't think running a daily sync will kill your disks. It might take quite a while with that many files to google drive.

You might consider doing an rclone check once a month which will check the md5sums locally and remote and report any differences.

Thank you so much.

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.