Understanding backup-dir behavior with box (and others)

Help and Support
1

Hello all, I’ve been doing a large amount of copy and sync to box.com - working fine - what I realized today is this

when using copy or sync + backup-dir - it behaves as you would expect, if a file is updated, the previous version is moved to the specified directory - however this prevents box.com’s (and I imagine other supported services) from leveraging their file versioning -TL;DR is you have the current and 1 previous version of the file (plus files actually deleted) in backup-dir

when running copy without --backup-dir updated files are copied “replace existing” and create a box version (which is what I would like to do)

so the question is - what would be the right way to only move actual deletions to backup-dir ? run a check and to a move on what’s missing locally?

thanks

2

Is that because rclone effectively does this to update an existing file

* rm backup-dir/file
* mv dest/file  backup-dir/file
* upload dest/file

Is there something else rclone could be doing?

I think you are saying that you’d like --backup-dir to only handle files that get deleted, not files which get updated? That means you can use the box versioning.

Don’t box keep your deleted files though in the trash? I’m wondering whether this is redundant?

3

just wondering if there is something different I can do as is, anything deleted / renamed I want to archive, so in that senses backup-dir works fine, I am not sure what the optimal solution is, because for versioning, copy DOES behave correctly with versions.

I was sort of going this direction doing a check + copy and check + delete (with backup dir)

4

btw I just realized who replied to me :wink: so yes I think (at least for box) simply doing a “copy” which seems to update to the latest version and keeping it in it’s current location would be ideal for my use case

it seems like right now to achieve that it would be a sequential copy/check/delete combo to achieve the same effect?

ps while box does have a trash can, even on our enterprise account there is no SLA on how long things stay in trash (without paying even more) nor is the trash well organized so I do not consider it reliable.

thanks

5

Well what you could do is go one step futher with --backup-dir and use a dated backup-dir, so one for every day that you run rclone. This will give you your archive of versions. The advantage of this technique is that it will work with other providers that don’t do versions.

So do an rclone copy /src box: then an rclone sync /src box: --backup-dir box:

I think that would work. It isn’t super efficient as it does two traversals but it would work. There is also a small window between the sync and copy where files could be updated on the src which would mean they wouldn’t get a version on the dst they’d get copied into backup dir instead.

That could potentially be another flag say --backup-dir-deletes-only or something like that.

OK!

b2 does this much better - when you delete a file it replaces it with a deleted marker, so you can see all the old versions if necessary, but the file doesn’t exist normally.

6

thanks! the box trash can is more on par with how windows recycle bin works, very disorganized

7

I think even with just doing 1 copy + sync cycle - for any file that changes often you would see the current version in it’s normal location and the versions only since the last sync in the backup-dir

I think without doing dated subfolders in the --backup-dir it’s more of a matter how far back you want to go, so relatively infrequent sync perhaps.

I think a deletes only option would be ideal, thanks

8

Can you please make a new issue on github about that with a link to this thread please and we can put it on the list!

Thanks