Safe (size based) backup

[Terry4567]

So I had an SSD failing with ext4, (due to server overheating)

file sizes randomly showing up as 0bytes, I wonder how would rclone have handled that.
I guess based on modification date it wouldn't have overwritten the good backup with 0byte file.

Nevertheless, I will always use --min-size 1B (although doc says smallest unit is K)

But I was wondering it might be beneficial to have a switch to prevent overwritting files with 0bytes, thus empty files are still copied over.

Unless there is something similar already.

[asdffdsa (jojothehumanmonkey)]

are you familiar with --size-only ?

@Terry4567

although doc says smallest unit is K

where did your read that?
the docs mention that suffix of B for Byte

@Terry4567

But I was wondering it might be beneficial to have a switch to prevent overwritting files with 0bytes, thus empty files are still copied over.

sorry, not sure that that means, can you share a detailed example?

[Terry4567]

are you familiar with --size-only ?
I think checking the modification time is safer in this case.

@asdffdsa

where did your read that?

--min-size - Don't transfer any file smaller than this
Controls the minimum size file within the scope of an rclone command. Default units are KiB but abbreviations K, M, G, T or P are valid.

Rclone Filtering

No B here.

sorry, not sure that that means, can you share a detailed example?
Sure thing,
C:\cv.doc 1KB // day to day SSD drive, gets corrupted, new size will be 0bytes
B:\cv.doc 1KB // backup

C: drive gets corrupted, C:\cv.doc turns into 0byte length when read,
so to protect against the fault migrating to backups we could add a switch to prevent overwriting existing files in B:\ with zero length ones.

(but still allow to copy empty files if they are not seen on backup yet)

Data can be very expensive, and if the auto backup tool helps to destroy backups that's not good.

[asdffdsa (jojothehumanmonkey)]

@Terry4567

No B here.

ok, good point. actually, looks like the docs need at least two edits.
you are welcome to edit the doc at github. if not, i can do it?

1. https://rclone.org/filtering/#min-size-don-t-transfer-any-file-smaller-than-this
2. https://rclone.org/filtering/#max-size-don-t-transfer-any-file-larger-than-this

@Terry4567

auto backup tool helps to destroy backups that's not good

one workaround, that my backup scripts always uses is --backup-dir using a datetime stamp for the dir.

@Terry4567

switch to prevent overwriting existing files in B:\ with zero length ones.
(but still allow to copy empty files if they are not seen on backup yet)

what other tool already implements that feature?

[Terry4567]

Not sure how can I edit the doc.

Yeah, versioning is a solution, just needs redundant space.

I didn't see this feature anywhere else, I guess not many people had partially failed disks. I'm happy with --min-size 0B, it's just not the best solution.

[asdffdsa (jojothehumanmonkey)]

@Terry4567

Not sure how can I edit the doc.

click the pencil icon at https://github.com/rclone/rclone/blob/master/docs/content/filtering.md

another possible approach.

1. use rclone to get list of zero byte files.
2. feed that to rclone copy --exclude-from

@Terry4567

I didn't see this feature anywhere else, I guess not many people had partially failed disks. I'm happy with --min-size 0B, it's just not the best solution.

rclone, like rsync, is just a copy tool, with some extra features.
on windows, for real backup and recovery tool, i use veeam.