Rclone fails to copy to GCS-bucket with retention policy

I thought so. The question then would be, which variable is set (with one after each backup changed file) for the likewise changing destination folder?

You'd do this in the shell - in bash something like

remote:backup-`date -Is`

Great, so it should work.

The problem with veeam (not rclone) could then be whether the respective old chain data files also "match" the current remaining veeam-backup files.

If anyone has already tested, I would be grateful for any advice.

First of all many thanks for the (as always) great help, ncw.

hello,
i use veeam agent and veeam backup and replication with rclone and backup to wasabi, which is s3 compatible.

please ask and i will try to answer...

Hm, seems not to work:

"Error: unknown shorthand flag: 's' in -s`"

i use the the rclone flag --backup-dir.
each time i run rclone, it creates an archive folder and copies that veeam .vbm file to the archive folder.

Could you please write the complete command here?

sure, give me a couple of minutes

i have a python script that generates the rclone commands
notice that the new backup files go to a folder named backup and any files like the .vbm that would be overwritten are moved into an archive folder with the current date.time.
the key flag is --backup-dir

here is the full command, take a look at it and ask questions.

DEBUG : rclone: Version "v1.49.1" starting with parameters ["C:\data\rclone\scripts\rclone.exe" "sync" "v:\EN07\en07.veaamfull\EN07" "wasabiwest01:vserver03-en07.veaamfull\backup\" "--stats=0" "--progress" "--backup-dir=wasabiwest01:vserver03-en07.veaamfull\archive\20190920.170411\" "--log-level" "DEBUG" "--log-file=C:\data\rclone\logs\en07.veaamfull\20190920.170411\en07.veaamfull_20190920.170411_rclone.log"]

Should not there be a variable under ...\archive**2019.**.. ?

not sure what you mean about variable, do you mean python variables?

maybe the text i gave you is confusing as it is the output from the log file.

here is the raw command that can be run directly on the command line.

rclone.exe sync "v:\EN07\en07.veaamfull\EN07" wasabiwest01:vserver03-en07.veaamfull\backup\ --stats=0 --progress --backup-dir=wasabiwest01:vserver03-en07.veaamfull\archive\20190920.170411\ --log-level DEBUG --log-file=C:\data\rclone\logs\en07.veaamfull\20190920.170411\en07.veaamfull_20190920.170411_rclone.log

variable = placeholder

By that I meant that the log extract refers only to the specific time (...archive\20190920.170411... ) .

The command to set the date and time for each new folder to be created would have to be different.

there is no placeholder in that command. that command is run as is.
i have a python script that generates the command on the fly and then runs the command.
the python scripts does a bunch of other stuff as emailing the rclone logs to me and so on.

i am not a linux guy but i know it is easy to use placeholders in bash using date and time.

the main thing i want to share is the use of --backup-dir.

also, do not freak about those .vbm files. if you lost the file or it becomes corrupted, it does not matter too much. i have dealt with corrupted .vbm files before.

Sorry, but I am not a python script guy :slight_smile: , anyway:

does it not mean that in the backups chain metadata files (as the name says) the chains are included? So if I want to restore an old chain, do I not need the matching old metadata file?

correct, the medadata is not needed at all.
i just wanted to let you know that if somehow you messed up lost a .vbm file while testing rclone, that veeam backup and replication, you can just manually import the backup .vib and .vbk files.
you can contact veeam tech support or post a question to the veeam forums.

also, i noticed that in your rclone command you might be using an rclone encrypted backend.
if so there is no need as veeam itself can encrypt it own files. no need to double encrypt...

good luck and let us know how we can help.

and if you know veeam software, there is no way to to have
and if you want to push the limits of veeam and rclone, check this out.
veeam backup and replication has an instant vm recovery option.

well, you can rclone mount a folder in the cloud and then veeam instant vm recovery on it.

and if you contact veeam tech support, they will tell you unless you pay them a fortune for veeam cloud connect, you cannot do that. and they will tell you that the next version of veeam, 10, you will be able to instant recovery from cloud storage.

but rclone mount can get it done now on the cheap!

Great, did not know about that. I will give that a try, thanks.

Hm, when it comes to encryption I always have my difficulties trusting the closed systems of the manufacturers. Veeam says they encrypt with AES 256bit, but not with what it does.

One veeam-seller recently offered us the enterprise plus version for two VMs for about 7500, - € !

So you're damn right :wink:

imho, veeam is expensive but they offer great tech support.
you do not need cloud connect to backup to cloud.
and perhaps you do not need the enterprise edition.
the first year i paid veeam for enterprise editon and after that i started to use the free edition and rclone for cloud backup