Sync Duplicacy backup

Help and Support
1

What is the problem you are having with rclone?

I'm trying to sync my Windows local Duplicacy backup on a usb flash drive. Tried running the command in an elevated Powershell.

mkdir D:\test works if done manually.

Run the command 'rclone version' and share the full output of the command.

rclone v1.65.0

  • os/version: Microsoft Windows 11 Enterprise 22H2 (64 bit)
  • os/kernel: 10.0.22621.2861 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.21.4
  • go/linking: static
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Local Windows filesystem.

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

Not using remote backend.

A log from the command that you were trying to run with the -vv flag

rclone sync --progress -vv C:\Users\xxx\duplicacy_local D:\duplicacy_local
2023/12/20 21:01:20 ERROR : chunks/ff/eccda285a364369e043ac5db747942eb1e8b3cf15a903fe5cc7fd17e62beb3: Failed to copy: mkdir \\?\D:: Access is denied.
2023/12/20 21:01:20 ERROR : Local file system at //?/D:/duplicacy_local: not deleting files as there were IO errors
2023/12/20 21:01:20 ERROR : Local file system at //?/D:/duplicacy_local: not deleting directories as there were IO errors
2023/12/20 21:01:20 ERROR : Attempt 3/3 failed with 2821 errors and: mkdir \\?\D:: Access is denied.
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:              2821 (retrying may help)
Elapsed time:        14.8s
2023/12/20 21:01:20 INFO  :
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:              2821 (retrying may help)
Elapsed time:        14.8s

2023/12/20 21:01:20 DEBUG : 5 go routines active
2023/12/20 21:01:20 Failed to sync with 2821 errors: last error was: mkdir \\?\D:: Access is denied.
2

  1. Can you post full log and not only redacted part? You can make it shorter by trying to sync only some small subfolder.

  2. What is your D drive? What file system?

  3. Please run rclone -vv mkdir d:\newdirectory and post output here. And if it works try also rclone -vv mkdir d:\duplicacy_local\newdirectory

3

Sure:

PS C:\Users\xxx> rclone sync --progress -vv C:\Users\xxx\duplicacy_local\snapshots D:\duplicacy_local
2023/12/21 15:33:07 DEBUG : rclone: Version "v1.65.0" starting with parameters ["C:\\Users\\xxx\\rclone\\rclone.exe" "sync" "--progress" "-vv" "C:\\Users\\xxx\\duplicacy_local\\snapshots" "D:\\duplicacy_local"]
2023/12/21 15:33:07 DEBUG : Creating backend with remote "C:\\Users\\xxx\\duplicacy_local\\snapshots"
Enter configuration password:
password:
2023/12/21 15:33:18 DEBUG : Using config file from "C:\\Users\\xxx\\.config\\rclone\\rclone.conf"
2023/12/21 15:33:18 DEBUG : fs cache: renaming cache item "C:\\Users\\xxx\\duplicacy_local\\snapshots" to be canonical "//?/C:/Users/xxx/duplicacy_local/snapshots"
2023/12/21 15:33:18 DEBUG : Creating backend with remote "D:\\duplicacy_local"
2023/12/21 15:33:18 DEBUG : fs cache: renaming cache item "D:\\duplicacy_local" to be canonical "//?/D:/duplicacy_local"
2023/12/21 15:33:18 DEBUG : home/1: Need to transfer - File not found at Destination
2023/12/21 15:33:18 DEBUG : home/2: Need to transfer - File not found at Destination
2023/12/21 15:33:18 DEBUG : Local file system at //?/D:/duplicacy_local: Waiting for checks to finish
2023/12/21 15:33:18 DEBUG : Local file system at //?/D:/duplicacy_local: Waiting for transfers to finish
2023/12/21 15:33:18 ERROR : home/2: Failed to copy: mkdir \\?\D:: Access is denied.
2023/12/21 15:33:18 ERROR : home/1: Failed to copy: mkdir \\?\D:: Access is denied.
2023/12/21 15:33:18 ERROR : Local file system at //?/D:/duplicacy_local: not deleting files as there were IO errors
2023/12/21 15:33:18 ERROR : Local file system at //?/D:/duplicacy_local: not deleting directories as there were IO errors
2023/12/21 15:33:18 ERROR : Attempt 1/3 failed with 2 errors and: mkdir \\?\D:: Access is denied.
2023/12/21 15:33:18 DEBUG : home/1: Need to transfer - File not found at Destination
2023/12/21 15:33:18 DEBUG : home/2: Need to transfer - File not found at Destination
2023/12/21 15:33:18 DEBUG : Local file system at //?/D:/duplicacy_local: Waiting for checks to finish
2023/12/21 15:33:18 DEBUG : Local file system at //?/D:/duplicacy_local: Waiting for transfers to finish
2023/12/21 15:33:18 ERROR : home/1: Failed to copy: mkdir \\?\D:: Access is denied.
2023/12/21 15:33:18 ERROR : home/2: Failed to copy: mkdir \\?\D:: Access is denied.
2023/12/21 15:33:18 ERROR : Local file system at //?/D:/duplicacy_local: not deleting files as there were IO errors
2023/12/21 15:33:18 ERROR : Local file system at //?/D:/duplicacy_local: not deleting directories as there were IO errors
2023/12/21 15:33:18 ERROR : Attempt 2/3 failed with 2 errors and: mkdir \\?\D:: Access is denied.
2023/12/21 15:33:18 DEBUG : home/1: Need to transfer - File not found at Destination
2023/12/21 15:33:18 DEBUG : home/2: Need to transfer - File not found at Destination
2023/12/21 15:33:18 DEBUG : Local file system at //?/D:/duplicacy_local: Waiting for checks to finish
2023/12/21 15:33:18 ERROR : home/1: Failed to copy: mkdir \\?\D:: Access is denied.
2023/12/21 15:33:18 DEBUG : Local file system at //?/D:/duplicacy_local: Waiting for transfers to finish
2023/12/21 15:33:18 ERROR : home/2: Failed to copy: mkdir \\?\D:: Access is denied.
2023/12/21 15:33:18 ERROR : Local file system at //?/D:/duplicacy_local: not deleting files as there were IO errors
2023/12/21 15:33:18 ERROR : Local file system at //?/D:/duplicacy_local: not deleting directories as there were IO errors
2023/12/21 15:33:18 ERROR : Attempt 3/3 failed with 2 errors and: mkdir \\?\D:: Access is denied.
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 2 (retrying may help)
Elapsed time:        12.0s
2023/12/21 15:33:18 INFO  :
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 2 (retrying may help)
Elapsed time:        12.0s

2023/12/21 15:33:18 DEBUG : 5 go routines active
2023/12/21 15:33:18 Failed to sync with 2 errors: last error was: mkdir \\?\D:: Access is denied.
PS C:\Users\xxx>

D drive is an USB flash drive with NTFS file system.

PS C:\Users\xxx> rclone -vv mkdir D:\newdirectory
2023/12/21 15:35:48 DEBUG : rclone: Version "v1.65.0" starting with parameters ["C:\\Users\\xxx\\rclone\\rclone.exe" "-vv" "mkdir" "D:\\newdirectory"]
2023/12/21 15:35:48 DEBUG : Creating backend with remote "D:\\newdirectory"
Enter configuration password:
password:
2023/12/21 15:35:54 DEBUG : Using config file from "C:\\Users\\xxx\\.config\\rclone\\rclone.conf"
2023/12/21 15:35:54 DEBUG : fs cache: renaming cache item "D:\\newdirectory" to be canonical "//?/D:/newdirectory"
2023/12/21 15:35:54 DEBUG : Local file system at //?/D:/newdirectory: Making directory
2023/12/21 15:35:54 ERROR : Attempt 1/3 failed with 1 errors and: mkdir \\?\D:: Access is denied.
2023/12/21 15:35:54 DEBUG : Local file system at //?/D:/newdirectory: Making directory
2023/12/21 15:35:54 ERROR : Attempt 2/3 failed with 1 errors and: mkdir \\?\D:: Access is denied.
2023/12/21 15:35:54 DEBUG : Local file system at //?/D:/newdirectory: Making directory
2023/12/21 15:35:54 ERROR : Attempt 3/3 failed with 1 errors and: mkdir \\?\D:: Access is denied.
2023/12/21 15:35:54 DEBUG : 2 go routines active
2023/12/21 15:35:54 Failed to mkdir: mkdir \\?\D:: Access is denied.
PS C:\Users\xxx>

This is true also with an elevated Powershell prompt.

Out of rclone it works:

PS C:\Users\xxx> mkdir D:\newdirectory

    Directory: D:\

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d----          21/12/2023    15:37                newdirectory
1 Like
4

i can reproduce, seems to be a bug with latest rclone
https://github.com/rclone/rclone/issues/7468

use an older version of rclone.
or
in these topics, i share a couple of workarounds.
https://forum.rclone.org/t/copying-multiple-folders-across-different-drives-partitions-on-windows/43387/10?u=asdffdsa
https://forum.rclone.org/t/1-65-fails-listing-local-folders/43148/7?u=asdffdsa

2 Likes
5

Now we need Windows expert... but I think we have all details needed to crack it.

6

Latest beta - v1.66 - is already built with go1.21.5 so could do until v1.65.1 is released

1 Like
7

Yes, worked with latest beta, thanks!

Speaking of, do you have any suggested parameters i could use to improve the sync of my Duplicacy backup to the flash drive?

8

Rclone seems unnecessary for a local transfer. Have you tried something like robocopy?

9

First time i hear of it. Duplicacy backups can change when pruning chunks so a sync felt suitable in order to have an exact copy of the dir over time.

10

welcome to the forum,

imho, robocopy has a fatal flaw, does not verify file transfers using checksums.
so not a good choice for backups.

11

I normally combine it with something like digup to check the content hasn't changed digup - A Digest Updating Tool - panthema.net

12

Dumb question, i use duplicacy backup -hash in order to detect file differences by hash (rather than size and timestamp).

Rclone sync documentations says it uses size and modification time or MD5SUM.

After it says

--checksum                                    Check for changes with size & checksum (if available, or fallback to size only).

So it's only size and not also timestamp?

Anyway looks like implementing --checksum for my rclone sync is a good idea, what do you think?

13

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.