Rclone to wasabi with veeam endpoint

the .vbm changes per backup and you need to copy that wasabi.
it keeps a list of all the backups files, and some other stuff

on wasabi, you should enable versioning for all files.

note: the.vbm is not critical as you can perform a restore without it.

if you want to get fancy, on the recovery media, you can have a copy of rclone and whatever scripts you might need.
in a disaster where your computer is down. you can download the files and perform a restore.

i have a usb key i keep in my wallet for disaster recovery.
among other neat features, it can boot ws2019-hyperv with veeam backup and replication installed and enough storage to download the veeam files onto.

Thanks so much that all makes sense. I have just been running the copy command on the folder and it finds any changes and copies them up so that's perfect. I suppose I just now need to be careful rclone does not copy the files while a backup is running. This is where your wiki on shadow copy should solve it im guessing. Not sure if I will need your other switches like Immutable etc I will need to spend more time alone (lucky me) :slight_smile: Really appreciate all your help on this its been so invaluable. thank you :slight_smile:

immutable, that is the most important flag of all!

if your internet connection if fast enought, no need for vss.
write a simple script to run a veeam backup and then rclone copy

OK I need to understand immutable next :slight_smile: I'm on it. That recovery usb key sounds something special. I gotta learn that one day! Cheers

yeah, the usb key is special.
that i can plug it into any computer and boot wserver, w10, 20+ .iso files and more..

cheers

Morning :slight_smile:

can you clarify one thing for me about the --immutable switch? As the backups are using incremental forever and the immutable switch stops the .vmb file from being copied as it changes each day. Surely in my case, this is a bad thing as the copy in wasabi becomes outdated compared to the other backup files that are getting copied? Maybe in your scenario, it's different, or I am missing something. Thanks

i run rclone twice for each veeam backup
rclone source dest --immutable --exclude=*.vbm
rclone source dest --include=*.vbm

can i use the exclude switch on folders as well? I mean will it effect the files within the folders?

filters can be applied to folders as detailed here

to test and understand filtering; use rclone ls source

Cheers for this and all your help. Do you run those commands one after the other from the same batch command or do you use to separate cmd files? Thanks

one batch file.

  1. run vagent
  2. runs the two rclone commands.
  3. sends the log as an email.
  4. bunch of other stuff.

Thanks, I am up and running with all your help. Thanks a lot for everything. I don't suppose you have any pointers on how to set up the email alerts do you?

I also notice that you say that you run the veeam agent, do you do this from a script as I have never seen that. I just schedule the agent within windows?

sure, glad yo help.

you can start a veeam backup like this, which is based on the schedule.
"C:\Program Files\Veeam\Endpoint Backup\Veeam.EndPoint.Manager.exe" /backup
more options here

as for sending email is use this and a command looks like, which it has a gui, to create and test sending email and then you can run it command the command line
this is generated from a python script, but you can see the basic idea.
cmd = f'{ScriptsDir}\\swithmail.exe --silent --XML "swithsettings.xml" --Subject "Backup {BackupNameFull} {TheBackupStatus}" --to "email@domain.com"'

Thought I would just add a little something regarding VSS in case you still want to pursue that further. I have also made use of @asdffdsa 's VSS wiki, it has proved invaluable! I would just add that the use of the -nw flag for the vshadow.exe invocation has some implications. No writers shadow copies have some limitations, so you can read up and determine whether those limitations would affect you.

hi,

i am thinking of updating that wiki, if you have any suggestions, please let me know.

that is a good point about -nw.

in my use case, i do not need it.
at the locations i manage, i run veeam backup software, which has explicit support for VSS writers and for recovery of data backed up by the writers such as sql servers, exchange servers and so on.

i have a python script that runs 7zip, fastcopy, veeam and then uses VSS and rclone to get the files created from a local backup server to cloud. also using rclone to run check, cryptcheck and so on.

Thanks @asdffdsa , yeah that would be my only suggestion for now, to explain the -nw flag which is used in the wiki. I also think that flag would usually not be used so that the writers can do their thing. From my very limited understanding, that would be the more resilient way of doing it.

in what specific use-case do you require?

  • rclone
    and
  • vss without -nw

Quick question if I may. With regard to retention period on the Veeam agent. Currently I am choosing 7 days as I have limited storage space on the backup drive. However after 7 days the main vbk file gets modified again and rclone sees this and over writes the wasabi file with another huge backup file taking hours / days. How are you dealing with that? the first 6 days are great as the vib files are small but once the vbk gets rolled up on the 7th things don't go as I expected....

it depends on how you configure vagent.

does vagent create a new .vbk or modify an existing .vbk?