EDIT: Script updated to 1.3 to correct the 4 issues mentioned by asdfdsa later down. Now 1.3 for more fixes and improvements...
Well, since you knew the magic word (BOTH of them in fact!!) then I must oblige...
Here you go, all done! (now where do I send the bill? 
)
Script has been tested, but of course I recommend you also test it thoroughly yourself and see that everything works as you expect before putting it into a live environment. Also check that you feel the formatting of time/date is to your liking. Note that " : " (normally in time formats) is an illegal character on google drive and most filesystems so I had to replace that. I opted for human readability over easy computer parsing.
Description:
Makes a "mirror backup" on the cloud of your the folder you designate as the the source. This includes deletion of files, ie. if you delete something locally then it gets removed from the "current backup" also.
However, these files are not lost. Instead of being deleted or overwritten they are moved to the "archive" and timestamped. Thus you have a full "back in time" archive for all file revisions ever made in case you should need to recover anything - either due to malware, errors, mistakes or dead harddrives.
The script does not clean up old archive data automatically, so you may want to remove the oldest stuff manually once a year or something (but it is not really required when your storage is unlimited, up to you...).
The system is efficient and does not require any duplicate data to be stored.
New entries in the archive only happen if anything actually changed. Otherwise nothing needs to be done and rclone will just exit after checking everything is up to date.
Usage:
- Change the settings so they are correct for you (ask if needed but I commented it pretty robustly I think)
- Use task scheduler to schedule the script to run as often as you wish. For example once every hour, but that is totally up to you. There is little to no waste of computer resources to run it often if you wish. Ask if you need help with that.
sidenote1:
If the files you want to backup&archive include files with weird or strict permissions that your own user can not normally access then you may want to run the script as SYSTEM account instead of yours. However, if you do then we may need to check that rclone uses the right config. I don't expect this will be needed, but I mention it for the sake of completeness.
sidenote2:
Google drive has a maximum upload limit of 750GB/day. That means that you may initially run into that limit and probably need several days to get up to speed. From there it should not be a problem as you most likely won't add or change 750GB pr day. Hitting the upload limit is the only circumstance under which rclone will fail do perform the backup&archive (which is not much we can do about really...)
It won't require any restart or interaction on your part though. It will keep running as scheduled even if that happened.
@asdffdsa Feel free to give this a once-over to see if I missed anything important
archivesync.bat
:: Archivesync v1.3 by Stigma, credit to asdfdsa(Jojo) for pseudocode assistance, debugging and emotional support ;)
@echo off
::Before we start, get a file-lock on this script to prevent concurrent instances
call :getLock
exit /b
:getLock
:: The CALL will fail if another process already has a write lock on the script
call :main 9>>"%~f0"
exit /b
:main
:: Body of script goes here. Only one process can ever get here
:: --------------SETTINGS START ----------------
:: set this to the folder where rclone.exe is located on your system
set "rclonepath=C:\rclone"
:: Set this to the folder (or driveletter) you want to protect with backup and archive
set "sourcepath=F:\testsource"
:: Set this the the folder (usually on a remote) you want to save the backup and archive to (does not necessarily have to be an rclone remote)
set "destpath=TD5C1:\scripttest\CurrentBackup"
:: Set this to the folder (usually on a remote) which will contain old "deleted" or "overwritten" revisions of files. I suggest keeping it next to your backup folder but it could be put anywhere.
set "archivepath=TD5C1:\scripttest\archive"
:: Set this path to where you want the logfile to be made.
set "logfilepath=F:\logfiles"
:: Set the detail of logging - from least verbose to most: ERROR or NOTICE or INFO or DEBUG (default, NOTICE, is usually sufficient)
:: see documentaion for more info : https://rclone.org/docs/#log-level-level
set "loglevel=INFO"
:: Set any other non-essential flags you want rclone to use. Can leave as empty set "flags=" if you want none
set "flags=--fast-list --progress --drive-chunk-size 64M"
::------------------SETTINGS END------------------
::----------------MAIN SCRIPT START --------------
::Various timestamp formatting in separate function (you can change timestamp there if you want).
call :FORMATTIME
::Make the logfile directory if it doesn't already exist
if not exist "%logfilepath%\" mkdir "%logfilepath%"
echo Archivesync is starting, stand by ...
echo rclone is gathering listing from the destination. This may take a minute if there is a lot of data.
echo:
:: Now let us sync. This makes a mirror of sourcepath to destpath (including removing files if required), and any files that get "overwritten" or "deleted" as a
:: result from destpath, will be moved into archive and and timestamped instead - effectively creating a full archive of all revisions of files you have ever had.
%rclonepath%\rclone sync "%sourcepath%" "%destpath%" %flags% --backup-dir="%archivepath%\%date%" --log-file="%logfilepath%\%date%.log" --log-level=%loglevel%
echo:
::If exit code of above command was anything but normal, display an error and pause
if not %ERRORLEVEL% equ 0 (
echo rclone reported an error during the sync. Check that settings are correct. Check rclone logfile for spesific info about the error.
exit /b 1
) else (
echo Sync completed sucessfully!
exit /b 0
)
::----------------MAIN SCRIPT END -----------------
::--------------HELPER FUNCTONS START--------------
:FORMATTIME
for /f "usebackq skip=1 tokens=1-6" %%g in (`wmic Path Win32_LocalTime Get Day^,Hour^,Minute^,Month^,Second^,Year ^| findstr /r /v "^$"`) do (
set day=00%%g
set hours=00%%h
set minutes=00%%i
set month=00%%j
set seconds=00%%k
set year=%%l
)
set month=%month:~-2%
set day=%day:~-2%
set hh=%hours:~-2%
set mm=%minutes:~-2%
set ss=%seconds:~-2%
:: This can be easily modified to your liking if you prefer another timestamp format (for archive and logs) - credit to asdfdsa(Jojo)
set date=%year%.%month%.%day%_%hh%.%mm%
exit /b 0
::----------------HELPER FUNCTONS END--------------