Rclone with VSS (ShadowCopy)

Help and Support
1

What is the problem you are having with rclone?

Running rclone from a Powershell script with admin rights it dosn't copies any file from source directory, but if I run it from Shell it works.
It's important that the source directory is a mounted Shadow Copy (VSS) on my computer. This way I can backup some files that are currently used on my computer, like Outlook's file (.PST and .OST).
If I run rclone's command in my Powershell script executed as Administrator, the copy fails. But if I run rclone's command directly into Powershell console it works, always as Admininistrator.
I found on some forums that Windows blocks some file types to be Shadow Copy, as Outlook.OST files, but I have already fixed that.

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.2

  • os/version: Microsoft Windows 11 Pro 22H2 (64 bit)
  • os/kernel: 10.0.22621.1105 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.18.6
  • go/linking: static
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Local storage on PC.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

C:\Temp\rclone.exe sync C:\_ShadowCopy\Users\parasect\AppData\Local\Microsoft\Outlook\ C:\Temp\RSYNC_ADMiN\ --log-file=C:\temp\rclone_log.txt

The rclone config contents with secrets removed.

Default Config, no changes made

A log from the command with the -vv flag

2023/01/16 15:22:33 DEBUG : rclone: Version "v1.59.2" starting with parameters ["C:\\Temp\\rclone.exe" "sync" "C:\\_ShadowCopy\\Users\\parasect\\AppData\\Local\\Microsoft\\Outlook\\" "C:\\Temp\\RSYNC_ADMiN\\" "--log-file=C:\\temp\\rclone_log_console.txt" "-vv"]
2023/01/16 15:22:33 DEBUG : Creating backend with remote "C:\\_ShadowCopy\\Users\\parasect\\AppData\\Local\\Microsoft\\Outlook\\"
2023/01/16 15:22:33 NOTICE: Config file "C:\\Users\\parasect\\AppData\\Roaming\\rclone\\rclone.conf" not found - using defaults
2023/01/16 15:22:33 DEBUG : fs cache: renaming cache item "C:\\_ShadowCopy\\Users\\parasect\\AppData\\Local\\Microsoft\\Outlook\\" to be canonical "//?/C:/_ShadowCopy/Users/parasect/AppData/Local/Microsoft/Outlook/"
2023/01/16 15:22:33 DEBUG : Creating backend with remote "C:\\Temp\\RSYNC_ADMiN\\"
2023/01/16 15:22:33 DEBUG : fs cache: renaming cache item "C:\\Temp\\RSYNC_ADMiN\\" to be canonical "//?/C:/Temp/RSYNC_ADMiN/"
...
2023/01/16 15:22:33 DEBUG : parasect@outlook.ost: Reopening on read failure after 0 bytes: retry 7/10: read \\?\C:\_ShadowCopy\Users\parasect\AppData\Local\Microsoft\Outlook\parasect@outlook.ost: The process cannot access the file because another process has locked a portion of the file.
2023/01/16 15:22:33 DEBUG : parasect@outlook.ost: Reopening on read failure after 0 bytes: retry 8/10: read \\?\C:\_ShadowCopy\Users\parasect\AppData\Local\Microsoft\Outlook\parasect@outlook.ost: The process cannot access the file because another process has locked a portion of the file.
2023/01/16 15:22:33 ERROR : backup.pst: Failed to copy: read \\?\C:\_ShadowCopy\Users\parasect\AppData\Local\Microsoft\Outlook\backup.pst: The process cannot access the file because another process has locked a portion of the file.
2023/01/16 15:22:33 DEBUG : parasect@outlook.ost: Reopening on read failure after 0 bytes: retry 9/10: read \\?\C:\_ShadowCopy\Users\parasect\AppData\Local\Microsoft\Outlook\parasect@outlook.ost: The process cannot access the file because another process has locked a portion of the file.
2023/01/16 15:22:33 DEBUG : parasect@outlook.ost: Reopening on read failure after 0 bytes: retry 10/10: read \\?\C:\_ShadowCopy\Users\parasect\AppData\Local\Microsoft\Outlook\parasect@outlook.ost: The process cannot access the file because another process has locked a portion of the file.
2023/01/16 15:22:33 DEBUG : parasect@outlook.ost: Reopen failed after 0 bytes read: failed to reopen: too many retries
2023/01/16 15:22:33 NOTICE: parasect@outlook.ost: Removing partially written file on error: read \\?\C:\_ShadowCopy\Users\parasect\AppData\Local\Microsoft\Outlook\parasect@outlook.ost: The process cannot access the file because another process has locked a portion of the file.
2023/01/16 15:22:33 ERROR : parasect@outlook.ost: Failed to copy: read \\?\C:\_ShadowCopy\Users\parasect\AppData\Local\Microsoft\Outlook\parasect@outlook.ost: The process cannot access the file because another process has locked a portion of the file.
2023/01/16 15:22:33 ERROR : Local file system at //?/C:/Temp/RSYNC_ADMiN/: not deleting files as there were IO errors
2023/01/16 15:22:33 ERROR : Local file system at //?/C:/Temp/RSYNC_ADMiN/: not deleting directories as there were IO errors
2023/01/16 15:22:33 ERROR : Attempt 3/3 failed with 2 errors and: read \\?\C:\_ShadowCopy\Users\parasect\AppData\Local\Microsoft\Outlook\parasect@outlook.ost: The process cannot access the file because another process has locked a portion of the file.
2023/01/16 15:22:33 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Errors:                 2 (retrying may help)
Checks:                80 / 80, 100%
Elapsed time:         0.1s

2023/01/16 15:22:33 DEBUG : 5 go routines active
2023/01/16 15:22:33 Failed to sync with 2 errors: last error was: read \\?\C:\_ShadowCopy\Users\parasect\AppData\Local\Microsoft\Outlook\parasect@outlook.ost: The process cannot access the file because another process has locked a portion of the file.
2

hello and welcome to the forum,

most of the errors in the log file are about .ost

trying to understand the rclone issue?

with your script, if you change rclone.exe to a simple file copy, does that also error out?
testing with something like
copy C:\_ShadowCopy\Users\parasect\AppData\Local\Microsoft\Outlook\parasect@outlook.ost c:\users\parasect\documents

fwiw, i have a wiki about using vss with rclone
https://github.com/rclone/rclone/wiki/How-to-enable-VSS-for-rclone

3

I'm sorry, I forgot some important information.
I have create a Powershell script that uses a Powershell module I found online to manage Shadow Copies.
The module works fine, also with other commands like xcopy (not Copy-Items by Powershell which returns same error as Rclone).

Since I can't attach files I link them LINK TO FILES

4

your script works perfectly everywhere else, except for .ost and .pst?

so with your same script, executed in the same way, xcopy on an .ost works?
by changing C:\Temp\rclone.exe sync to xcopy

5

never tried to copy a .ost before, but as a test, my own script was able.
note: msoutlook was open and in use

my script does the following

  1. create the vss snapshot,
  2. run the following command(s) against the snapshot.
    a. rclone
    b. fastcopy
    c. 7zip
    d. restic

for rclone, the command was

C:\data\rclone\rclone.exe sync "b:\rclone\vss\ost_20230116.104217\Users\user01\AppData\Local\Microsoft\Outlook" "C:\Users\user01\AppData\Local\Temp\rclone\DestDirRclone\ost/rclone/backup"   --stats=5h  --ask-password --config="C:\data\rclone\rclone.conf" --log-level=DEBUG --log-file="C:\data\rclone\logs\ost\20230116.104217\rclone.log"

and the output was

DEBUG : user@email.com.ost: md5 = dc1a3ffa18fb996293f59463154dc876 OK
INFO  : user@email.com.ost: Copied (new)
6

Hi,
I just wanted to mention that I've had several problems with Powershell and rclone (and none with CMD) and at least one of them the culprit is just PowerShell which eats up memory when piping something simple like: type A | rclone rcat remote: . Change the rclone part for other command and it eats memory as well.

I found that it also slows down transfers up to ten times compared to CMD.

I haven't investigated the issue further, but used CMD and call PowerShell from it just when needed.

Good luck
Manuel F

7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.