IO errors when sync needs to delete files

Help and Support
1

What is the problem you are having with rclone?

I've successfully run an initial backup, but it seems to error out when it needs to delete files from the remote.

Run the command 'rclone version' and share the full output of the command.

rclone v1.57.0
- os/version: Microsoft Windows 10 Pro 2009 (64 bit)
- os/kernel: 10.0.19044.1526 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.17.2
- go/linking: dynamic
- go/tags: cmount

Which cloud storage system are you using?

Local.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync e: encryptednas: --bwlimit 100m -P -vv

The rclone config contents with secrets removed.

[nasbackup]
type = alias
remote = Z:\Backup\

[encryptednas]
type = crypt
remote = nasbackup:
password = xxxxx

A log from the command with the -vv flag

The log is thousands of lines, this is the error I'm getting after it has finished the sync.

2022-02-20 18:40:17 DEBUG : Encrypted drive 'encryptednas:': Waiting for checks to finish
2022-02-20 18:40:17 DEBUG : Encrypted drive 'encryptednas:': Waiting for transfers to finish
2022-02-20 18:40:17 ERROR : Encrypted drive 'encryptednas:': not deleting files as there were IO errors
2022-02-20 18:40:17 ERROR : Encrypted drive 'encryptednas:': not deleting directories as there were IO errors
2022-02-20 18:40:17 ERROR : Attempt 3/3 failed with 2 errors and: not deleting files as there were IO errors

2022/02/20 18:40:17 DEBUG : 2 go routines active
2022/02/20 18:40:17 Failed to sync with 2 errors: last error was: not deleting files as there were IO errors
2

hi,
can you post the top 30 lines of the debug log,
and an example of an individual file with that error.

3

It does not show what files failed, or it probably does but I can't scroll that far up in the terminal.

All lines in the terminal are like this, with the only changes being file path. These are some of the last lines before it finshes.

2022-02-20 18:40:04 DEBUG : Programs/Benchmarks/Blender Open Data/blender-benchmark-1.0b2-windows64/blender/2.79/python/lib/test/test_import/data/circular_imports/basic.py: Size and modification time the same (differ by 0s, within tolerance 100ns)
2022-02-20 18:40:04 DEBUG : Programs/Benchmarks/Blender Open Data/blender-benchmark-1.0b2-windows64/blender/2.79/python/lib/test/test_import/data/circular_imports/basic.py: Unchanged skipping
2022-02-20 18:40:04 DEBUG : Programs/Benchmarks/Blender Open Data/blender-benchmark-1.0b2-windows64/blender/2.79/python/lib/test/test_import/data/circular_imports/indirect.py: Size and modification time the same (differ by 0s, within tolerance 100ns)
2022-02-20 18:40:04 DEBUG : Programs/Benchmarks/Blender Open Data/blender-benchmark-1.0b2-windows64/blender/2.79/python/lib/test/test_import/data/circular_imports/indirect.py: Unchanged skipping
2022-02-20 18:40:04 DEBUG : Programs/Benchmarks/Blender Open Data/blender-benchmark-1.0b2-windows64/blender/2.79/python/lib/test/test_import/data/circular_imports/basic2.py: Size and modification time the same (differ by 0s, within tolerance 100ns)
2022-02-20 18:40:04 DEBUG : Programs/Benchmarks/Blender Open Data/blender-benchmark-1.0b2-windows64/blender/2.79/python/lib/test/test_import/data/circular_imports/subpackage.py: Size and modification time the same (differ by 0s, within tolerance 100ns)
2022-02-20 18:40:04 DEBUG : Programs/Benchmarks/Blender Open Data/blender-benchmark-1.0b2-windows64/blender/2.79/python/lib/test/test_import/data/circular_imports/util.py: Size and modification time the same (differ by 0s, within tolerance 100ns)
2022-02-20 18:40:04 DEBUG : Programs/Benchmarks/Blender Open Data/blender-benchmark-1.0b2-windows64/blender/2.79/python/lib/test/test_import/data/circular_imports/rebinding2.py: Size and modification time the same (differ by 0s, within tolerance 100ns)
2022-02-20 18:40:04 DEBUG : Programs/Benchmarks/Blender Open Data/blender-benchmark-1.0b2-windows64/blender/2.79/python/lib/test/test_import/data/circular_imports/rebinding.py: Size and modification time the same (differ by 0s, within tolerance 100ns)
4

let's use a debug log, instead of spiting output to console.
`--log-level=DEBUG --log-file=c:\path\to\local\folder\rclone.log

rclone might not output much to the terminal so add
--progress

5

This is all "ERROR" log entries. The log is well over 100MB:

2022/02/20 20:57:45 ERROR : System Volume Information: failed to open directory "System Volume Information": open \\?\E:\System Volume Information: Access is denied.

2022/02/20 20:59:42 ERROR : Encrypted drive 'encryptednas:': not deleting files as there were IO errors
2022/02/20 20:59:42 ERROR : Encrypted drive 'encryptednas:': not deleting directories as there were IO errors
2022/02/20 20:59:42 ERROR : Attempt 1/3 failed with 2 errors and: not deleting files as there were IO errors

2022/02/20 20:59:42 ERROR : System Volume Information: failed to open directory "System Volume Information": open \\?\E:\System Volume Information: Access is denied.

2022/02/20 21:01:32 DEBUG : Encrypted drive 'encryptednas:': Waiting for checks to finish
2022/02/20 21:01:32 DEBUG : Encrypted drive 'encryptednas:': Waiting for transfers to finish
2022/02/20 21:01:32 ERROR : Encrypted drive 'encryptednas:': not deleting files as there were IO errors
2022/02/20 21:01:32 ERROR : Encrypted drive 'encryptednas:': not deleting directories as there were IO errors
2022/02/20 21:01:32 ERROR : Attempt 2/3 failed with 2 errors and: not deleting files as there were IO errors

2022/02/20 21:01:32 ERROR : System Volume Information: failed to open directory "System Volume Information": open \\?\E:\System Volume Information: Access is denied.

2022/02/20 21:03:28 DEBUG : Encrypted drive 'encryptednas:': Waiting for checks to finish
2022/02/20 21:03:28 DEBUG : Encrypted drive 'encryptednas:': Waiting for transfers to finish
2022/02/20 21:03:28 ERROR : Encrypted drive 'encryptednas:': not deleting files as there were IO errors
2022/02/20 21:03:28 ERROR : Encrypted drive 'encryptednas:': not deleting directories as there were IO errors
2022/02/20 21:03:28 ERROR : Attempt 3/3 failed with 2 errors and: not deleting files as there were IO errors
2022/02/20 21:03:28 INFO  : 
Transferred:   	    1.422 MiB / 1.422 MiB, 100%, 5.299 KiB/s, ETA 0s
Errors:                 2 (retrying may help)
Checks:            344513 / 344513, 100%
Transferred:            1 / 1, 100%
Elapsed time:      5m43.6s

2022/02/20 21:03:28 DEBUG : 2 go routines active
2022/02/20 21:03:28 Failed to sync with 2 errors: last error was: not deleting files as there were IO errors
6

sure, that is to be expected.
if the username used to run rclone cannot access a file/folder,
then rclone cannot access the file/folder.
use an --exclude filter

7

But that isn't my problem. rclone will not delete any files that is deleted from my machine.

It will happily sync new files but won't delete any files again off the remote.

8

well, it is your problem.
by default, rclone sync will not delete files if there are errors.

you can change that using
https://rclone.org/docs/#delete-before-during-after

9
Deleted

I see. From other file syncing programs I've used the synchronization option would always delete files on the mirrored side so I thought that and no duplication was the features separating clone and sync.

Just to be clear, the difference between sync and clone is that sync will delete and not make duplicates, but this didn't happen because of the Sys Vol errors?

Thanks for the help.

10

no such rclone command as clone???
please define your terms.

11

I meant copy not clone, sorry.

12

rclone copy
--- copy files
--- does not delete files
--- the dest does not match the source

rclone sync
--- copy files
--- delete files
--- the dest does match the source.

13

Even with the --delete-after flag rsync doesn't delete any files for me.

14

well, that is documented in the link i shared.

rclone

15

It still won't delete empty folders as there is IO errors but the files are gone. I can't get --exclude to work for "System Volume Information". Do you happen to know how --exclude works on Windows? There's only examples for Linux.

16

you need to read the entire webpage, it does explain filtering for windows,

anyhoo, filters can be confusing,
--exclude="/System Volume Information/"

1 Like
17

I missed the last /. Thanks.

18

rclone filtering is like that scene from the first matrix movie,

Morpheus leaps from building to building.
Neo tries and falls,
someone says "everyone falls the first time"
but in the end, Neo can stop bullets.

1 Like
19

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.