Trying to embed the date into the log file windows 11

Help and Support
1

What is the problem you are having with rclone?

I have gotten my backup to run, but there is one final finishing touch I am looking for. I would like to have the log file name include the date and time that the file was created.

Run the command 'rclone version' and share the full output of the command.

--- rclone v1.65.0

  • os/version: Microsoft Windows 11 Pro 24H2 (64 bit)
  • os/kernel: 10.0.26100.4061 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.21.4
  • go/linking: static
  • go/tags: cmount
No

Which cloud storage system are you using? (eg Google Drive)

P Cloud (mapped as network drive)
OneDrive (configured as OneDrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy C:\Users\User\Documents P:\Backup\User\documents --log-file=C:\Users\User\Documents\Computer\Backups\Data\PDrive_Docs_logfile.log -P

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[Google]
type = drive
client_secret = XXX

[OneDrive]
type = onedrive
token = XXX
drive_id = XXX
drive_type = personal

[pCloud]
type = pcloud
hostname = api.pcloud.com
token = XXX
### Double check the config for sensitive info before posting publicly

A log from the command that you were trying to run with the -vv flag

My intent is to save the processing as a .bat file and run through task scheduler. I have 2 .bat files to run, one for OneDrive and one for PCloud. I have attached the PCloud version here. My plan is to back up to Pcloud one night and OneDrive the next night scheduled to run at 11 PM, after I have gone to bed.

echo %date%
echo Docs Backup

:: Back up /local/documents
rclone copy C:\Users\User\Documents P:\Backup\User\documents --log-file=C:\Users\User\Documents\Computer\Backups\Data\PDrive_Docs_logFile.log -P -vv
::rclone copy "source_directory" "destination_directory" --suffix date +%Y%m%d_%H%M%S

:: Wait 60 seconds
timeout /t 60 /nobreak
echo
echo 60 seconds have passed.
echo

:: Back up /local/pictures
::rclone copy C:\Users\User\Pictures P:\Backup\User\Pictures --log-file=C:\Users\User\Documents\Computer\Backups\Data\PDrive_Pics_logfile.log -P
:: line commented out
echo Backup Complete
pause
::rclone copy --log-file=C:\Users\User\Documents\Computer\Backups\myPicslogfile.log -P

Sample of output file:
2025/05/21 19:15:19 DEBUG : rclone: Version "v1.65.0" starting with parameters ["rclone" "copy" "C:\Users\User\Documents" "P:\Backup\User\documents" "--log-file=C:\Users\User\Documents\Computer\Backups\Data\PDrive_Docs_logfile.log" "-P" "-vv"]
2025/05/21 19:15:19 DEBUG : Creating backend with remote "C:\Users\User\Documents"
2025/05/21 19:15:19 DEBUG : Using config file from "C:\Users\User\AppData\Roaming\rclone\rclone.conf"
2025/05/21 19:15:19 DEBUG : fs cache: renaming cache item "C:\Users\User\Documents" to be canonical "//?/C:/Users/User/Documents"
2025/05/21 19:15:19 DEBUG : Creating backend with remote "P:\Backup\User\documents"
2025/05/21 19:15:19 DEBUG : fs cache: renaming cache item "P:\Backup\User\documents" to be canonical "//?/P:/Backup/User/documents"
2025/05/21 19:15:19 NOTICE: My Music: Can't follow symlink without -L/--copy-links
2025/05/21 19:15:19 NOTICE: My Pictures: Can't follow symlink without -L/--copy-links
2025/05/21 19:15:19 NOTICE: My Videos: Can't follow symlink without -L/--copy-links
2025/05/21 19:15:19 DEBUG : Applebee’s_®_eGift_Card_2025-02-20.pdf: Modification times differ by -99.9885ms: 2025-02-20 08:16:03.0999885 -0500 EST, 2025-02-20 08:16:03 -0500 EST
2025/05/21 19:15:19 DEBUG : ARCHIVE.ZIP: Size and modification time the same (differ by 0s, within tolerance 100ns)
2025/05/21 19:15:19 DEBUG : ARCHIVE.ZIP: Unchanged skipping
2025/05/21 19:15:19 DEBUG : desktop.ini: Modification times differ by -447.4055ms: 2025-01-10 22:19:31.4474055 -0500 EST, 2025-01-10 22:19:31 -0500 EST
2025/05/21 19:15:19 DEBUG : SpeedCalculations.xlsx: Modification times differ by -343.0384ms: 2024-08-07 14:49:58.3430384 -0400 EDT, 2024-08-07 14:49:58 -0400 EDT
2025/05/21 19:15:19 DEBUG : Transcript.pdf: Size and modification time the same (differ by 0s, within tolerance 100ns)
2025/05/21 19:15:19 DEBUG : Transcript.pdf: Unchanged skipping
2025/05/21 19:15:19 DEBUG : vulcan_peace_sign_by_akatsuki1289_d2i1snw-fullview.jpg: Modification times differ by -445.0295ms: 2025-02-02 20:55:13.4450295 -0500 EST, 2025-02-02 20:55:13 -0500 EST
2025/05/21 19:15:19 DEBUG : Your Evaluation from M. Reynolds.msg: Size and modification time the same (differ by 0s, within tolerance 100ns)
2025/05/21 19:15:19 DEBUG : Your Evaluation from M. Reynolds.msg: Unchanged skipping
2025/05/21 19:15:19 DEBUG : desktop.ini: md5 = 833db2a588be3a47b2deff7e2d718408 OK
2025/05/21 19:15:19 DEBUG : SpeedCalculations.xlsx: md5 = 27fe07a3a73df6af320bc0e002195c4e OK
2025/05/21 19:15:19 DEBUG : Applebee’s_®_eGift_Card_2025-02-20.pdf: md5 = d0a286915c95ffabe4a10b76dd230844 OK
2025/05/21 19:15:19 DEBUG : vulcan_peace_sign_by_akatsuki1289_d2i1snw-fullview.jpg: md5 = c29224c9664a9b32c80fac36cafe1cce OK
2025/05/21 19:15:19 DEBUG : Books/July 26 2015.txt: Size and modification time the same (differ by 0s, within tolerance 100ns)
2025/05/21 19:15:19 DEBUG : Books/July 26 2015.txt: Unchanged skipping
2025/05/21 19:15:19 DEBUG : Airlines/311-brochure.pdf: Size and modification time the same (differ by 0s, within tolerance 100ns)
2025/05/21 19:15:19 DEBUG : Airlines/311-brochure.pdf: Unchanged skipping
2025/05/21 19:15:19 DEBUG : Airlines/311-poster.pdf: Size and modification time the same (differ by 0s, within tolerance 100ns)
2025/05/21 19:15:19 DEBUG : Airlines/311-poster.pdf: Unchanged skipping
2025/05/21 19:15:19 DEBUG : Artwork/073709-120118.pdf: Size and modification time the same (differ by 0s, within tolerance 100ns)
2025/05/21 19:15:19 DEBUG : Artwork/073709-120118.pdf: Unchanged skipping
2025/05/21 19:15:19 DEBUG : Artwork/383709-120118.pdf: Size and modification time the same (differ by 0s, within tolerance 100ns)
2025/05/21 19:15:19 DEBUG : Artwork/383709-120118.pdf: Unchanged skipping
2025/05/21 19:15:19 DEBUG : Airlines/311-credit-card.pdf: Size and modification time the same (differ by 0s, within tolerance 100ns)
2025/05/21 19:15:19 DEBUG : Airlines/311-credit-card.pdf: Unchanged skipping
2025/05/21 19:15:19 DEBUG : Airlines/OneNote Table Of Contents.onetoc2: Size and modification time the same (differ by 0s, within tolerance 100ns)
2025/05/21 19:15:19 DEBUG : Airlines/OneNote Table Of Contents.onetoc2: Unchanged skipping
2025/05/21 19:15:19 DEBUG : Books/July 26 2015.pdf: Size and modification time the same (differ by 0s, within tolerance 100ns)
2025/05/21 19:15:19 DEBUG : Books/July 26 2015.pdf: Unchanged skipping
2025/05/21 19:15:19 DEBUG : Camera_JPG/2024_10_20_147_TransportMuseum.jpg: Modification times differ by -981.8623ms: 2024-10-30 16:50:09.9818623 -0400 EDT, 2024-10-30 16:50:09 -0400 EDT

I tried several versions of configuration, but could not get any to work.

2 
@for /f "tokens=2 delims==" %%I in ('wmic os get localdatetime /format:list') do set datetime=%%I
@set datetime=%datetime:~0,4%%datetime:~4,2%%datetime:~6,2%.%datetime:~8,2%%datetime:~10,2%%datetime:~12,2%

rclone lsd . -vv --log-file=rclone.%datetime%.log
rclone cat rclone.%datetime%.log

rclone lsd . -vv --log-file=rclone.20250521.203620.log

rclone cat rclone.20250521.203620.log 
DEBUG : rclone: Version "v1.69.2" starting with parameters ["d:\\data\\rclone\\rclone.exe" "lsd" "." "-vv" "--log-file=rclone.20250521.203620.log"]
DEBUG : Creating backend with remote "."
DEBUG : Using config file from "d:\\data\\rclone\\rclone.conf"
DEBUG : fs cache: renaming cache item "." to be canonical "//?/d:/data/rclone/other/date"
DEBUG : 2 go routines active

and should rclone selfupdate

i meet mr.spock, captain kirk and of course uhura, more than once, at star trek conventions in new york city.

3

OneDriveDocsLogFile.log (498.6 KB)

I have signed pictures from James Doohan and Marina Sirtis when they were in Rochester.

MrSpock

4

nice!

  1. rclone self update
  2. recreate OneDrive:
  3. test again.

can't copy - source file is being updated (size changed from 1646 to 2596))
well, rclone, like most programs, will have an issue with files changing during the copy command.
will lead to corrupted backups.

the solution is to create an on-the-fly windows vss snapshot and use that snapshot as the source for the copy command.
luckily, i have a handy, dandy howto|wiki, shows how to do that with .cmd in four lines of code.

5

I find the explanation a wee bit confusing. I will take a look at it and see what I can do.

MrSpock

6

So what would the changes required be? I am a little confused as to how to integrate the changes into the .bat files. Could you outline the changes? Would the VS.cmd and exec.cmd be triggered from the .bat files?

Mike

7

i run a modfifed version of the wiki for 5+ years, on many windows server and desktop.
well over 20,000 snapshots created.

  1. download vshadow.exe. that is explained at the end of the wiki.

  2. create the files in step 1. and step 2.
    tho, for safe testing, i would change
    rclone sync c:\snapshot\data\ remote:data
    to
    rclone lsd c:\snapshot\data\ -vv

  3. run vs.cmd

  4. may your backups live long and propser ;wink

8

I'm looking to do a copy, instead of a sync. Would it still work with a copy command?

9

it works with any command line program.
including but not limited to 7zip, fastcopy, rclone, restic and even rsync

here is a snippet from falconer, my client+server backup programs written in python
in each case, the backup creates a vss snapshot, used as the source for all commands.

on client machine, copies keepass to backup server over tailscale

cmd01=!PING|ts-credserver01
cmd02=7z       a         "${zip_fn}"  "!SOURCEDIR"       ${7z_flags}  >  "!LOGDIRL/zip.log"
cmd03=7z       t         "${zip_fn}"  "!SOURCEDIR"       ${7z_flags}  >> "!LOGDIRL/zip.log"
cmd04=fastcopy /cmd=sync  ${fc_flags} "!SOURCEDIR\*"     /to="!DESTDIR\fastcopy\current"
cmd05=rsync    --backup               "!SOURCEDIR/"          "!DESTDIR/rsync/current" --backup-dir="!DESTDIR/rsync/archive/!DATETIME"    ${rsync_flags}
cmd06=rclone   check                  "!SOURCEDIR"           "!DESTDIR/rsync/current"                                                    ${rk05_rclone} ${rg_flags}
cmd07=!SIGNAL|{"name":"vserver03_en_rcloner_idrive_keepass", "server":"ts-vserver04", "job_user":"administrator"}

once the files are on the backup server, copies those files to multiple cloud servers.

cmd01=rclone copy  "!SOURCEDIR"   "${wasabi}/current" --backup-dir="${wasabi}/archive/!DATETIME"  --include="keepass*.7z" --max-age=${maxage}  --immutable    ${rg_flags}
cmd02=rclone check "!SOURCEDIR"   "${wasabi}/current"                                             --include="keepass*.7z" --max-age=${maxage}  ${rk02_rclone} ${rg_flags}
cmd03=rclone sync  "!SOURCEDIR"   "${wasabi}/current" --backup-dir="${wasabi}/archive/!DATETIME"  --exclude="keepass*.7z" --max-age=${maxage}                 ${rg_flags}
cmd04=rclone check "!SOURCEDIR"   "${wasabi}/current"                                             --exclude="keepass*.7z" --max-age=${maxage}  ${rk04_rclone} ${rg_flags}
cmd05=rclone sync  "${fsource}"   "${onedr}/current"  --backup-dir="${onedr}/archive/!DATETIME"                                                        ${rflags_onedr}
cmd06=rclone check "${fsource}"   "${onedr}/current"                                                                                    ${rk06_rclone} ${rflags_onedr}
10

Does vshadow need to be in the path?

11

no. can put vshadow.exe anywhere you want.

in fact, in my case, vshadow.exe is not found anywhere on the entire c: drive of the clients' machines.
it is stored on the credential server which runs linux, not windows.
and vshadow.exe is run remotely over tailscale on the client machine.

actually, rclone.exe, 7zip.exe, restic.exe, fastcopy.exe, rsync.exe are not on the client machines.
the execuables are stored on and run remotely from that linux server.

here is a snippet from a backup log. notice that the fullpath to vshadow.exe is
\\ts-credserver01\agent_en10_ro\vshadow.exe

20250522.190656:vss_source_dir='c:\\users\\user01\\keepass':self.snapshot_mountdir='b:\\falconer\\vss\\keepass_20250522.190654':sourcedir_relative='users\\user01\\keepass':source_drive_tobe_shadowed='c:'
20250522.190656:self.vss_source_dir='b:\\falconer\\vss\\keepass_20250522.190654\\users\\user01\\keepass':self.vss_mount_point='b:\\falconer\\vss':self.snapshot_mountdir='b:\\falconer\\vss\\keepass_20250522.190654'
20250522.190656:vshadow_command=cmd.exe /c start "exec_20250522190654" /MIN \\ts-credserver01\agent_en10_ro\vshadow.exe -nw -script=\\ts-credserver01\agent_en10_rw\backups\keepass\logs\20250522.190654\setvar-vshadow.cmd -exec=\\ts-credserver01\agent_en10_rw\backups\keepass\logs\20250522.190654\exec.cmd c:
12

I downloaded and installed the winsdksetup.exe which is suppose to include vshadow, but when I went to the command prompt and tried to run vshadow it didn't recognize the command.

13

search the entire c: drive

14

The other part that confuses me is about the multiple files. The .bat file needs to call the vshadow command, and then reference the new location?

15

only need to run one command,
c:\path\to\vshadow.exe -nw -script=setvars.cmd -exec=exec.cmd c:

but for convenience, i put it in vs.cmd

should use .cmd instead of .bat

16

Right now I am trying to get the vshadow command to run. The path to vshadow has a space in two spots. I am running Windows 11. If memory serves an ' is needed, but where?

17

Got over that hurdle ....... hang on while I catch my breath ..............

C:\Users\User\Documents\Computer\Backups>"C:\Program Files (x86)\Windows Kits\10\bin\10.0.26100.0\x64\Vshadow.exe" -nw -script=setvars.cmd -exec=exec.cmd c:

VSHADOW.EXE 3.0 - Volume Shadow Copy sample client.
Copyright (C) 2005 Microsoft Corporation. All rights reserved.

(Option: No-writers option detected)
(Option: Generate SETVAR script 'setvars.cmd')
(Option: Execute binary/script after shadow creation 'exec.cmd')
(Option: Create shadow copy set)

ERROR: COM call "CreateVssBackupComponents(&m_pVssObject)" failed.

  • Returned HRESULT = 0x80070005

  • Error text: Access is denied.

  • Please re-run VSHADOW.EXE with the /tracing option to get more details

18

sure,

  • so, all the .cmd files are on the same dir, correct?

  • post the contents of setvars.cmd and exec.cmd

not sure what the exact issue is yet, but from the wiki:
The vs.cmd must be run with administrator privileges.

19

So it will run with just one file? Using multiple files confuses me. I am enclosing the one file I am running. I had to rename it to a .txt file to get it to upload.

RClone_PCloud_Settings.txt (664 Bytes)

20

The result is running with Admin privileges.

Thank you for your assistance.

MrSpock