Webdav Incremential backup + schedule

Help and Support
1

Hello there

I'm on Windows server and I need to backup a distant webdav server (600GB and millions of files). I'd like to do a full backup first time during the weekend from friday 9PM til monday 06:00AM, then every night copy only new/modified files and do versionning (keep 20 oldest versions).

What's the best and easiest way to do it with rclone ?

Thank you
GM

2

hello and welcome to the forum,

rclone sync with --backup-dir using a timestamp.
the timestamp would have to be created with some kind of simple script

a basic example of a first run
rclone sync webdav: c:\path\to\backup.folder\current --backup-dir=c:\path\to\backup.folder\archive\20210303

for the daily run, add --min-age=24h

take a few minutes to read the docs and let me know if you have any questions.

3

Hello
Thank you for your reply.
Does this works also in copy mode instead of sync ?
Where should I add --min-age=24h ?

The webdav has thousands of folders containing files, the best for me would be to have a full copy and inside each folders, the versionned files with different timestamp, is it possible ?

Thank you

4

copy will copy files from source to dest, will not delete files in the dest.
sync will copy files from source to dest, will delete files in the dest

--min-age is added to the rclone command.

you should be able to use --suffix

5

Yes but where ? at the end of the full line ?

OK but how can i use it ? I need to use the backup-dir command too ?

Could you gave me the full command maybe ?

Thanks a lot

6

does not matter where the flags are positioned.

i have never used --suffix, do not want to have the old files in the same folder are the newest file.

if you want to test a command and you are not sure what will happen,
use --dry-run and/or --interactive

7

Ok will try this, thanks for your help

8

Just FYI, webDAV does not support modtime. So if you change a file but you do not change the size (not super likely but could happen), rclone will not see it.

This doesn't address your question but it is good to be aware of details like this.

9

excellent point, that would mean my suggestion about --min-age could be a ticking time :bomb:
https://www.youtube.com/watch?v=q_vGN9R3c3k

of course, a test recovery of a simulated disaster for a proposed backup solution, would catch that.

to clarify, it depends on the particular webdav server.
https://rclone.org/webdav/#modified-time-and-hashes

10

I'm nearly there, except for the date, I ran :

rclone -v copy Source: E:\Destination --suffix -revision-".$(date +%F)" --suffix-keep-extension

It did copied the modified file and kept the old file but I cant get the date, instead I get :
Filename-revision-.$(date +%F).docx

Any idea what is the windows command to get the date ?

Thank you

11

not a linux expert, perhpas try

 -revision-.`date +%F`
12

Thank you, I'm on windows not linux :slight_smile:
I'm trying this tonight : -revision-"%date%"

13

that will not work.

are you using
dos batch file?
powershell?
WSL?

14

dos batch file yes

15

I'm adding this to my .bat :
FOR /F "TOKENS=1 eol=/ DELIMS=/ " %%A IN ('DATE/T') DO SET dd=%%A
FOR /F "TOKENS=1,2 eol=/ DELIMS=/ " %%A IN ('DATE/T') DO SET mm=%%B
FOR /F "TOKENS=1,2,3 eol=/ DELIMS=/ " %%A IN ('DATE/T') DO SET yyyy=%%C
SET datestr=%dd%%mm%%yyyy%

rclone -v copy source: E:\destination --suffix -revision-"%datestr%" --suffix-keep-extension

16

i find that if you are going to store more than one year, to use.
easier to view in a file manager
SET datestr=%yyyy%%mm%%dd%

1 Like
17

Ok will post here tomorrow if this worked.

Thank you for your help

1 Like
18

Hi
Ok it works fine now :ok_hand:
One last thing, what is the command to end the job when the full scan is done ?

Thanks

19

what scan, i did not see a rclone scan command?

the rclone copy command will copy the files and then exit on it own

20

yes i mean rclone copy sorry, but it doesnt exit when copy is done, it keeps trying to copy error files.