Rclone to wasabi with veeam endpoint

Hi Everyone

I have been reading many posts and it looks like asdffdsa may be the person to help me out here if possible please? I am using Veeam Endpoint for my servers and various sites and it works great, however i would like to utilise wasabi to backup the Veeam files to wasabi each night. Also, I would like to utilise the shadow copies (all windows machines) so when the backup files change each night they can still get to wasabi without corruption. I am completely new to rclone and do not have any python experience but am keen to learn new things if you could help me implement these features It would mean a lot. I have managed to connect rclone into wasabi but this is as far as I have got as there are a lot switches and I would like to do it the proper way.
Thanks

Adam

What is your rclone version (output from rclone version)

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Which cloud storage system are you using? (eg Google Drive)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Paste command here

The rclone config contents with secrets removed.

Paste config here

A log from the command with the -vv flag

Paste  log here

hello and welcome to the forum,

  • tell me about your veeam setup, you do use vbar, veeam backup and replication or just vagent?
    free or paid?
    I use the free version of vbar and the free version of vagent.
  • about vss, start here
  • do you know about this.
    as far as i know, this is not documented. this can be reduced from 90 days to 30 days. first upload some veeam backup files, contact wasabi, tell them that you want the 30 day policy as you are using veeam.
  • i keep the most recent backups of veeam and other stuff in wasabi. and i keep a copy in aws s3 deep glacier.
  • do you use a local backup server, if so what do you use.
    i use the free windows server 2019 edition, with softraid 5 on refs file system with checksum integrity enabled for files.

Hi thanks for the prompt reply.

Tell me about your veeam setup, you do use vbar, veeam backup and replication or just vagent?
free or paid? Mainly just the veeam agent but occasionally comunity edition of

  • do you use a local backup server, if so what do you use.
    i use the free windows server 2019 edition, with softraid 5 on refs file system with checksum integrity enabled for files. I have hyper v 2019 server as well but with samsung ssd's in a hardware raid 10. I have a lot of physical servers as well though so I assume rclone needs to be installed on each of these separately?

to clarify. i run the **free** edition of windows server, the hyper-v edition on real physical hardware, not inside a virtual machine. not a evaluation version but free to use forever. i have a bunch of them in use at different locations.
image

yes, rclone needs to be installed on each machine, but you can create one rclone.conf and copy to other machines. does not matter if the machine runs windows, linux, termux on android, etc...

Ah yes I understand. I will have occasions when I use the same but for now lets assume its running on a normal 2016 / 2019 server or a windows 10 box. so any help very much appreciated

Adam.

this would be an example of a rclone command i use in production to copy veeam files to wasabi or aws s3 deep glacier.
it runs on wserver and w10.

@set RcloneCopyFilesCmd=C:\data\rclone\scripts\rclone.exe copy "b:\vssmount\rcloner\en07_wasabi01_veeam_br_en0701_20210419.122859\BR_EN07_02" "wasabi01:vserver03.veeam.br.en0701/en07/rclone/backup" --immutable --stats=0 --fast-list --bind=192.168.62.233 --s3-chunk-size=256M --s3-upload-concurrency=8 --exclude=*.vbm --log-level=DEBUG --log-file=C:\data\rclone\logs\en07_wasabi01_veeam_br_en0701\20210419.122859\rclone.log

1 Like

Thanks for this. I think I need to spend a lot of time reading as I have no clue what all those switches mean. Thank you for sharing though.

actually, to backup a folder with veeam backup files, i use two commands.

the command i posted is for .vib and .vbk files, not .vbm.
once uploaded the local file and the corresponding file should never change.
if a local file does not match the wasabi file, rclone will error, for example,
ERROR : VeeamAgentUser3bef6c4c-360f-11b2-a85c-9070b4e81f40/ABP_EN07_-_en07.local_1/ABP_EN07 -2021-04-11T091956.vbk: Source and destination exist but do not match: immutable file modified

very important to know that, so i have a script that scans the logs for errors.

however, the .vbm files changes each time veeam backup is run.
so i run a similar command without --immutable and add --include=*.vbm

1 Like

Thanks again in it simplest form, if I have a veeam folder say on D:\veeam can i just backup this folder to wasabi and then subsequently each night just backup the changes with that veeam folder? Simple and affective? Or am i missing something? Then if I ever lose the local drive I have a mirrored copy in the cloud to fall back on? That's all I need really...

Thanks

yes, the is basically correct.

what would be the best command to use to do this simply, but utilise shadow copy on windows drives. Just mirror the local backup drive exactly each night? So in event of a failure I can download the files again locally and perform bare metal recovery.

Thanks

did you read my wiki about vss?

I did but struggling to get my head around it as I am so new to this in the command line. Some of it makes sense but not all. will go over it again and see if I can get my head around it. Is this just the vss part? Then you need the rclone part after? Sorry for all the questions... I need to spend a few days with rclone alone. :slight_smile:

you are lucky, to be alone. that is a dream of mine.

as a half-monkey, half-human, living in a mutualistic symbiotic relationship, i am never really alone.
sure, i can ignore my human half, but he is such a noisy caveman.

What do you mean by this, do you mean the vib file never changes and a new one gets created?

once created, the .vib and the .vbk never change.

but new ones are created right? If the local copy failes can the files be copied back to the local system easy enough? The whole folder is wasabi, ie /veeambackups?

each time veeam is run, it will create a new backup file.
.vbk for each full and .vib for each incremental.

yes I have been testing it. I understand what you mean now thank you. How would I copy the folder back to the local system in event of failure?

I have just copied the files back, works like a dream. One thing puzzles me as the vbm file changes every time why is it not OK to back this up and overwrite the cloud version? Is this only a problem if a new backup chain gets created? Great software really loving it now I am understanding it further. Can an email of success somehow be scripted as well? Thanks for all your help on this. Really appreciated.