Well, I'm using rclone to backup a MiniO to another MiniO. I would like to know how I do the full backup and the incremental backup? the MiniO has the bucket versioning option, but when I send to the other MiniO (it also has the versioning option turned on) it only sends the latest version of a file. I wanted to know how to send all versions.
hello and welcome to the forum,
pretty sure rclone cannot do that.
do you know of any tool that can?
can mc?
does rclone always do incremental backup?
yes, but depends on what you mean by incremental
rclone sync /home/username/source remote:full --backup-dir=remote:incrementals/`date +%Y%m%d.%I%M%S` -vv --dry-run
as for rclone support for versions, check out these flags.
might write a script to fake a full backup including versions.
--s3-versions
--s3-version-at
As mentioned above you can use the --s3-versions flag to see all the versions and this will make a backup of all the objects. You'd want to enable this for the source but not the destination so
rclone sync source,versions:bucket dest:bucket
However this won't create versions on the destination, it will save the objects with suffixes like test-v2016-07-04-141032-000.txt
Rclone can't currently turn those names back into versions (--s3-versions is read only at the moment) and after a quick look through the s3 API I'm not sure it is actually possible.
i agree.
aws offers add-on product that will do that sort of replication.
@Bruna_Maia, might post at minio forum, ask about minio replication.
I think I didn't explain my problem correctly. I have a development MiniO and a backup MiniO, I'm using rclone to do this file transfer. My point is: during the day an X file can be changed several times, at the end of the day when the backup runs (I set it for night), this X file will go to the MiniO backup, but the intermediate versions will not. I already took the test.
But, I would like these versions in the MiniO backup, but these versions only go to the backup if I give the rclone command from each change in the X file, which is infeasible.
So I would like to do the following, an incremental backup every day, and a full backup once a week. Is it possible to do this with rclone?
For example, in the second one I changed the X file 10 times, but only the tenth version goes to the MiniO backup when the script runs. The next day, this file is changed 10 more times, and the tenth version goes to the MiniO backup, but that way I lose the tenth version of the second, thus having only the tenth version of the third. How could I solve this problem? I'm really confused
--- rclone cannot copy older versions from source to dest.
--- rclone can copy only the latest version from source to dest, which is the behavior you see.
--- maybe, you can fake it with a script using the flags mentioned above.
never heard of a tool that can do what you want.
so if you find it, please let us know.
Try this
rclone sync source,versions:bucket dest:bucket
It will preserve all the old versions just not as S3 versions. Using --track-renames would be a good idea too
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.