I need help creating versioning with crypt to dated folders

So, for all intents and purposes, I can simply redirect it to Z: instead of the unmounted rclone drive, and it should work?

Just using the mounted z: drive as the destination got past the logfile error.
I wonder if we can setup a temporary mount, for Z: to be mounted, before the backup job, just to resolve this throughout other pc's I'm backing up.
like rclone mount CPUPRO-Crypt: Z:
and go on with life.
The drive should eject, as soon as the job finishes, shouldn't it?

Well yes, that would work too - but almost certainly you have just malformed settings here...
It would be much better to just find out what is wrong with them.

Is your cloud bucket-based? Because otherwise you are missing a \ after the : in the path

What does rclone say if you just manually run
rclone ls CPUPRO-Crypt:Backup
?

Possible yes, but it's solving a very small problem with a cannon...
If you follow my troubleshooting steps we will have this solved in 5 minutes
It's almost certainly just a slightly wrong syntax in your path, and we can test that easily (see above)

Right now, I'm using rclone browser, for the drive mount.
CPUPRO-Crypt: is my encrypted rclone drive.
I have the backup, archive and logfiles, stored in that.

Here's my code, thus far.

:: Archivesync v1.3 by Stigma, credit to asdfdsa(Jojo) for pseudocode assistance, debugging and emotional support :wink:
@echo off

::Before we start, get a file-lock on this script to prevent concurrent instances
call :getLock
exit /b

:getLock
:: The CALL will fail if another process already has a write lock on the script
call :main 9>>"%~f0"
exit /b

:main
:: Body of script goes here. Only one process can ever get here

:: --------------SETTINGS START ----------------

:: set this to the folder where rclone.exe is located on your system
set "rclonepath=C:\Windows\System32"

:: Set this to the folder (or driveletter) you want to protect with backup and archive
set "sourcepath=C:\Users\Charles"

:: Set this as the folder (usually on a remote) you want to save the backup and archive to (does not necessarily have to be an rclone remote)
set "destpath=CPUPRO-Crypt:Backup\CurrentBackup"

:: Set this to the folder (usually on a remote) which will contain old "deleted" or "overwritten" revisions of files. I suggest keeping it next to your backup folder but it could be put anywhere.
set "archivepath=CPUPRO-Crypt:Backup\archive"

:: Set this path to where you want the logfile to be made.
set "logfilepath=Z:Backup\logfiles"

:: Set the detail of logging - from least verbose to most: ERROR or NOTICE or INFO or DEBUG (default, NOTICE, is usually sufficient)
:: see documentaion for more info : https://rclone.org/docs/#log-level-level
set "loglevel=INFO"

:: Set any other non-essential flags you want rclone to use. Can leave as empty set "flags=" if you want none
set "flags=--fast-list --progress --drive-chunk-size 64M"

::------------------SETTINGS END------------------
::----------------MAIN SCRIPT START --------------

::Various timestamp formatting in separate function (you can change timestamp there if you want).
call :FORMATTIME

::Make the logfile directory if it doesn't already exist
if not exist "%logfilepath%" mkdir "%logfilepath%"

echo Archivesync is starting, stand by ...
echo rclone is gathering listing from the destination. This may take a minute if there is a lot of data.
echo:

:: Now let us sync. This makes a mirror of sourcepath to destpath (including removing files if required), and any files that get "overwritten" or "deleted" as a
:: result from destpath, will be moved into archive and and timestamped instead - effectively creating a full archive of all revisions of files you have ever had.
%rclonepath%\rclone sync "%sourcepath%" "%destpath%" %flags% --backup-dir="%archivepath%%date%" --log-file="%logfilepath%%date%.log" --log-level="%loglevel%"
echo:

::If exit code of above command was anything but normal, display an error and pause
if not %ERRORLEVEL% equ 0 (
echo rclone reported an error during the sync. Check that settings are correct. Check rclone logfile for specific info about the error.
exit /b 1
) else (
echo Sync completed sucessfully!
exit /b 0
)

::----------------MAIN SCRIPT END -----------------
::--------------HELPER FUNCTONS START--------------
:FORMATTIME

for /f "usebackq skip=1 tokens=1-6" %%g in (wmic Path Win32_LocalTime Get Day^,Hour^,Minute^,Month^,Second^,Year ^| findstr /r /v "^$") do (
set day=00%%g
set hours=00%%h
set minutes=00%%i
set month=00%%j
set seconds=00%%k
set year=%%l
)
set month=%month:~-2%
set day=%day:~-2%
set hh=%hours:~-2%
set mm=%minutes:~-2%
set ss=%seconds:~-2%

:: This can be easily modified to your liking if you prefer another timestamp format (for archive and logs) - credit to asdfdsa(Jojo)
set date=%year%.%month%.%day%_%hh%.%mm%

exit /b 0
::----------------HELPER FUNCTONS END--------------

I see from the earlier post that you use a google product.
Do you know if it is a Google drive ? (from for example a Gsuite?)
or is it some sort of Google Cloud storage?

The former and most common of the two is NOT bucket base, and if so then your syntax is wrong and is missing a \

Under this assumption the crrect path would be
CPUPRO-Crypt:\Backup

I use the business version of Google Drive, the Google File Sync product, with supposedly "unlimited" storage for 10 bucks a month.

Ok, then try using the syntax in my example above. That should be correct for google drives (and all other non-bucket based remotes)

Change that on all the relevant settings and give it a new go.

If you would rather test than guess then just do
rclone ls CPUPRO-Crypt:\Backup
in the command prompt (from the fodler where rclone is installed)
and if you get a listing of anything it means it is indeed working correctly.

that mounted z:, it might be no visible to the script.
as a test, i would modify the script, at the top

dir z:
exit

and see if you get a listing of the z:

I'll give that a go, as soon as this backup finishes.
So far, its worked.

These sorts of issues is exactly why we shouldn't seek to solve this problem by adding more complexity to it hehe...

I got a listing, and about 50 screen dumps of a million little files that probably didn't need to be backed up, but they are there.

Great! Then whatever the syntax you used for rclone is the same as you want to use for your script setting. (minus the "rclone ls" part of course)

Your previous setting was slightly different, but that can be enough that rclone gets confused.

Since you've apparently been backing up the whole OS that is indeed a gazillion little files that you arguably didn't need to backup (and it's not helping the performance either as small files transfer inefficiently). I would use one of of the suggestions above to do this cleaner in the future. It's certainly provides a lot beter organization too if you have just a windowsbackup image + your actual valuable data - rather than a mess of files and folders of varying levels of importance.

After all, if your OS got borked it's not like you could just copy all this data back to the harddrive to restore everything anyway... OSes are a bit more complex than that. A systemimage on the other hand can just be loaded in from a windows recovery USB or windows DVD to restore it as-it-was with minimal effort, so that's likely much more useful than trying to sort through this mess for the important bits after a disaster-recovery event.

Just backing up my user folder, in Microsoft. The whole OS, nope. Just c:/users/Charles.
I've been using Macrium Reflect for disc image based restores.
If you've got something easier to tangle into this batch file, feel free to send code. :smiley:

I wouldn't really suggest adding something like that to the script. You would likely not want a whole system-image to be created on each sync anyway.

But if you just schedule a system-image to be made every so-and-so interval and just put the put the output into one of the folders this script covers then rclone will just pick it up without needing to complicate this particular script further. That should be fairly trivial to do - and you can use whatever backup software you want to really...

  1. i am sure that Macrium can run a script after each system image backup; so have it run rclone sync on that folder.
  2. you can use VSS as well.
  3. when i have a set of folders containing a large amount of small files, i zip the folders and rclone them to cloud.
  4. why are you backing up driverfs folder? it is just a cache folder, not the actual data

I've used and still use Macrium on my personal stuff.
The Server license is a little steep, price wise, for what little I want to do with it.
I've got Veeam and Imperus Backup at my disposal, one is paid for, one is literally free.
Macrium is close to 300 bucks for the regular server and 500 for the plus edition, if memory serves me correctly.
I'm honestly looking to spend the absolute minimum on this, yet getting the most R.O.I. out of that low expenditure as possible.
Don't get me wrong, I love Macrium, but I'm wanting something low cost, that I can roll out, not only on my own stuff, but potentially on some of my friends and clients, using Google File Sync drives I have purchased, to store "just in case everything flies all to hell" style backups.
I have a personal backup using external drives and a local nas, but if a crypto virus or a fire breaks out, it would be comforting to know that my data is up in the cloud.
C.Y.A with backups. :smiley:

we have given you a lot of well tested options.

veeam agent and veam backup and replication, community editions, are very powerful, reliable and free for servers and desktops.

for sure, your nas can backup its data to the cloud.

no matter what software you use to backup your data,
@thestigma gave you a script to get your backup files to the cloud.

what else can we help you with?

I tried out the new code, and it just throws up this error.


I don't know if I'm not holding my mouth right or what.
Maybe you can find where my error is?
:: Archivesync v1.3 by Stigma, credit to asdfdsa(Jojo) for pseudocode assistance, debugging and emotional support :wink:
@echo off

::Before we start, get a file-lock on this script to prevent concurrent instances
call :getLock
exit /b

:getLock
:: The CALL will fail if another process already has a write lock on the script
call :main 9>>"%~f0"
exit /b

:main
:: Body of script goes here. Only one process can ever get here

:: --------------SETTINGS START ----------------

:: set this to the folder where rclone.exe is located on your system
set "rclonepath=C:\rclone"

:: Set this to the folder (or driveletter) you want to protect with backup and archive
set "sourcepath=W:"

:: Set this the the folder (usually on a remote) you want to save the backup and archive to (does not necessarily have to be an rclone remote)
set "destpath=CPUPRO-Crypt:Client_Backup\WCE\CurrentBackup"

:: Set this to the folder (usually on a remote) which will contain old "deleted" or "overwritten" revisions of files. I suggest keeping it next to your backup folder but it could be put anywhere.
set "archivepath=CPUPRO-Crypt:Client_Backup\WCE\archive"

:: Set this path to where you want the logfile to be made.
set "logfilepath=CPUPRO-Crypt:Client_Backup\WCE\logfiles"

:: Set the detail of logging - from least verbose to most: ERROR or NOTICE or INFO or DEBUG (default, NOTICE, is usually sufficient)
:: see documentaion for more info : https://rclone.org/docs/#log-level-level
set "loglevel=INFO"

:: Set any other non-essential flags you want rclone to use. Can leave as empty set "flags=" if you want none
set "flags=--fast-list --progress --drive-chunk-size 64M"

::------------------SETTINGS END------------------
::----------------MAIN SCRIPT START --------------

::Various timestamp formatting in separate function (you can change timestamp there if you want).
call :FORMATTIME

::Make the logfile directory if it doesn't already exist
if not exist "%logfilepath%" mkdir "%logfilepath%"

echo Archivesync is starting, stand by ...
echo rclone is gathering listing from the destination. This may take a minute if there is a lot of data.
echo:

:: Now let us sync. This makes a mirror of sourcepath to destpath (including removing files if required), and any files that get "overwritten" or "deleted" as a
:: result from destpath, will be moved into archive and and timestamped instead - effectively creating a full archive of all revisions of files you have ever had.
%rclonepath%\rclone sync "%sourcepath%" "%destpath%" %flags% --backup-dir="%archivepath%%date%" --log-file="%logfilepath%%date%.log" --log-level=%loglevel%
echo:

::If exit code of above command was anything but normal, display an error and pause
if not %ERRORLEVEL% equ 0 (
echo rclone reported an error during the sync. Check that settings are correct. Check rclone logfile for spesific info about the error.
exit /b 1
) else (
echo Sync completed sucessfully!
exit /b 0
)

::----------------MAIN SCRIPT END -----------------
::--------------HELPER FUNCTONS START--------------
:FORMATTIME

for /f "usebackq skip=1 tokens=1-6" %%g in (wmic Path Win32_LocalTime Get Day^,Hour^,Minute^,Month^,Second^,Year ^| findstr /r /v "^$") do (
set day=00%%g
set hours=00%%h
set minutes=00%%i
set month=00%%j
set seconds=00%%k
set year=%%l
)
set month=%month:~-2%
set day=%day:~-2%
set hh=%hours:~-2%
set mm=%minutes:~-2%
set ss=%seconds:~-2%

:: This can be easily modified to your liking if you prefer another timestamp format (for archive and logs) - credit to asdfdsa(Jojo)
set date=%year%.%month%.%day%_%hh%.%mm%

exit /b 0
::----------------HELPER FUNCTONS END--------------