I have finished my batch file for Windows for a more complex operation of rclone and I would like to share it in case someone is looking for a similar solution. Not sure where to put it, so I have chosen this forum.
The idea is: While rclone can backup a directory tree, it is a bit complex (or impossible?) to specify several directory tree. So you use a batch to execute several rclone commands one after another - easy so far.
But it would be nice to add a log file to review the whole operation - ok, I have done it. You need to download a wintee application - https://code.google.com/archive/p/wintee/downloads - and put it into the same directory as the batch and rclone. This allows he output to go on the screen AND in the log file.
Going on, I have also added a feature to use another file to log current progress, so if the execution is interrupted and then run again, the second run can skip rclone commands that were already done and go to the part where the execution was stopped last time.
Above all, since I use Amazon Cloud Drive and it was not working a few times, the batch will check that the Amazon AWS server is online. If not, the backup is cancelled (otherwise, the rclone is usually stuck waiting for it to go online). If you don"t need this part you can delete it from the Initialization section.
The set up is done near the “CONFIGURATION” and “PUT YOUR LINES HERE” lines. There are comments throughout the file,
If it is useful to you, enjoy!
@echo off
:: Created by Vit Kovalcik, 2017 (vit[at]pastel[dot]cz)
::
:: Features:
:: - Multiple subsequent rclone executions possible
:: - Logging on screen and into a log file named according to the current date and time
:: requires the "wtee" - https://code.google.com/archive/p/wintee/downloads
:: - Writing the progress notes to a separate file. So when a run is interrupted
:: it can skip the backups that has already completed in the previous run and continue approximately
:: from the point of interruption.
:: If the backups completes, the progress file is deleted and the next backups starts at the beginning again.
:: NOTE: This doesn't support paths with "^" character ... I gave up the Windows batch escaping hell.
:: - Checking if Amazon AWS server is online (if not, the backup is cancelled)
setlocal EnableDelayedExpansion
echo.
rem exit /b
:: CONFIGURATION
:: rc_operation can be "copy" or "sync" (without quotes)
SET rc_operation=sync
set last_run_file=rclone_last_incomplete_run.txt
call :Initialization
:: Exclamation mark character (!) has to be escaped like this (^!) inside quotes or like this (^^!) outside them
:: And if the "!" charater is used somewhere in the path, it is safer to use environment variables than to use the paths as call parameters.
:: (Safer because it is very hard to escape multiple "!" in the line properly... I have really tried, but failed so far.)
:: Example 1:
call :SingleBackup "My Lovely Directory" C:\MyLovelyDirectory secret:MyLovelyDirectory
:: Example 2:
set backup_info=Another Directory
set backup_params=--exclude /SomeSubdirectory/ C:\AnotherDirectory secret:AnotherDirectory
call :SingleBackup
:: Example 3:
set backup_info=C:\^^!MyDirectoryWithExclamationMark
set backup_params= C:\^^!MyDirectoryWithExclamationMark secret:^^!MyDirectoryWithExclamationMark
call :SingleBackup
:: PUT YOUR LINES HERE
call :Finalization
goto :eof
:Initialization
:: Get start time:
for /F "tokens=1-4 delims=:.," %%a in ("%time%") do (
set /A "start=(((%%a*60)+1%%b %% 100)*60+1%%c %% 100)*100+1%%d %% 100"
)
:: Setting name of the file for logging
if not exist "log" mkdir log
for /f "tokens=2 delims==" %%I in ('wmic os get localdatetime /format:list') do set datetime=%%I
set log_file=log\%datetime:~0,4%_%datetime:~4,2%_%datetime:~6,2%-%datetime:~8,2%_%datetime:~10,2%_%datetime:~12,2%.txt
echo Checking connection to the Amazon AWS... | wtee -a %log_file%
Ping www.amazonaws.com -n 1 -w 2000 1>nul 2>&1
if errorlevel 1 (
echo Not connected to the internet or Amazon is not working. | wtee -a %log_file%
exit /b
)
echo Connection OK. | wtee -a %log_file%
echo. | wtee -a %log_file%
set last_run_count=0
set last_run_current_line=0
set last_run_disable_write=0
if not exist %last_run_file% (
echo Previous backup completed successfully. | wtee -a %log_file%
echo. | wtee -a %log_file%
) else (
echo Reading last incomplete backup data to continue where we stopped. | wtee -a %log_file%
call :ReadLastRun
echo. | wtee -a %log_file%
)
goto :eof
:Finalization
:: All ok, so delete the file containg the progress
del /q %last_run_file%
rem Get the end time:
for /F "tokens=1-4 delims=:.," %%a in ("%time%") do (
set /A "end=(((%%a*60)+1%%b %% 100)*60+1%%c %% 100)*100+1%%d %% 100"
)
rem Get elapsed time:
set /A elapsed=end-start
rem Show elapsed time:
set /A hh=elapsed/(60*60*100), rest=elapsed%%(60*60*100), mm=rest/(60*100), rest%%=60*100, ss=rest/100, cc=rest%%100
if %mm% lss 10 set mm=0%mm%
if %ss% lss 10 set ss=0%ss%
if %cc% lss 10 set cc=0%cc%
echo. | wtee -a %log_file%
echo. | wtee -a %log_file%
echo ==================================== | wtee -a %log_file%
echo Finished in %hh%:%mm%:%ss% | wtee -a %log_file%
echo ==================================== | wtee -a %log_file%
goto :eof
:SingleBackup
:: First parameter = directory to be echo-ed
:: Other parameters are passed to rclone (without %rc_operation%)
:: If no parameters are passed, we will use %backup_info% and %backup_params% variables instead
for /f "tokens=2 delims==" %%I in ('wmic os get localdatetime /format:list') do set loc_datetime=%%I
set loc_datetime=%loc_datetime:~8,2%:%loc_datetime:~10,2%:%loc_datetime:~12,2%
set SB_info=%1
rem throw the first parameter away
shift
set params=%1
:SB_Params_loop
shift
if [%1]==[] goto SB_Params_afterloop
set params=%params% %1
goto SB_Params_loop
:SB_Params_afterloop
if "!SB_info!"=="" (
set SB_info=!backup_info!
set params=!backup_params!
)
echo. | wtee -a %log_file%
echo ************************************************* | wtee -a %log_file%
echo %loc_datetime% Backing up !SB_info! | wtee -a %log_file%
echo ************************************************* | wtee -a %log_file%
echo. | wtee -a %log_file%
if %last_run_current_line% GEQ %last_run_count% goto:SB_LastRun_WriteProgress
call set compared_line=%%last_run_lines[%last_run_current_line%]%%
if not !compared_line! == !params! (
rem [WARNING!!! The "rem" here CANNOT be replaced by "::" ]
rem Mismatch in the last run and current run -> Delete last run file and disable the last run check feature for curent run.
rem It will be "reseted" and the next run will start from the beginning and won't skip anything.
del /q %last_run_file%
set last_run_disable_write=1
set last_run_count=0
)
set /A temp=last_run_current_line+1
if %temp% LSS %last_run_count% (
rem This was already done AND it is not the last line in the last run's progress file,
rem so we can skip this backup
echo Skipping because it seems to have completed successfully in the last run. | wtee -a %log_file%
goto :SB_AfterBackup
)
:SB_LastRun_WriteProgress
set /A temp=last_run_current_line+1
if %temp% GTR %last_run_count% (
if %last_run_disable_write% NEQ 1 (
rem In the next line there must NOT be a space before >>
rem Also, we are escaping the "!" for "easier" file loading (with EnableDelayedExpasion)
rem TODO: This would have o be adjusted to support filenames with "^" character.
echo %params:!=^^^^^^^!%>> !last_run_file!
)
)
:SB_LastRun_Done
rem echo Parameters: !params! | wtee -a %log_file%
rclone !rc_operation! !params! 2>&1 | wtee -a %log_file%
rem timeout 3
:SB_AfterBackup
set /A last_run_current_line+=1
goto :eof
:ReadLastRun
set /A i=0
for /F "usebackq delims=" %%a in ("%last_run_file%") do (
rem Kind of embarrasing: I haven't found a way to simply load lines from that can possibly contain the "!" character
rem So I am escaping all the exclamation marks using "^!" when writing the file...
rem And at this point I am expecting that the file contents are already escaped (there is "^!" instead of simple "!")
call set "last_run_lines[%%i%%]=%%a"
call set /A last_run_count=%%i%%+1
set /A i+=1
)
rem for /L %%i in (0,1,%last_run_count% - 1) do call echo %%last_run_lines[%%i]%%
goto :eof