How to get rclone sync to delete remote files even if there are IO errors?


#1

Hi,

I am backing up user directories under Windows. As such I typically get the following errors:

failed to open source object: open \?\c:\users\public\ntuser.dat.LOG2: The process cannot access the file because it is being used by another process
2018/01/07 14:01:04 ERROR : : error reading source directory: CreateFile \?\c:\users\public\Recent: Access is denied.
2018/01/07 14:00:40 ERROR : Encrypted drive ‘encrypted:/public’: not deleting files as there were IO errors

How do I tell rclone sync that I want to delete remote files in spite of these errors?

Better, how do I suppress each kind of error? Specifically, I never want “process cannot access the file because it is being used by another process” to prevent remote deletes.

I also don’t know how to suppress the error for c:\users\public\Recent. I don’t actually understand what is wrong there as I seem to have read-access (from Explorer).

Any ideas?

Thanks,
Gili


#2

You can’t at the moment, but this is planned in this issue: https://github.com/ncw/rclone/issues/642

There isn’t a way of doing that at the moment. But what you can do is use --exclude to exclude the file names - that is what I’d do.

Is that a special directory of some kind? (not a Windows expert)


#3

There isn’t a way of doing that at the moment. But what you can do is use --exclude to exclude the file names - that is what I’d do.

I can’t predict which files will be locked ahead of time. It depends on what application the user has open at a given time.

I also don’t know how to suppress the error for c:\users\public\Recent. I don’t actually understand what is wrong there as I seem to have read-access (from Explorer).

Is that a special directory of some kind? (not a Windows expert)

It’s a Windows “junction” which is similar to hard links on Linux: https://stackoverflow.com/a/9042600/14731

It looks like rclone can’t access any junctions, even though it’s running under my user and I have no problem accessing these junctions manually. Do you have a Windows box (even in a VM) to reproduce/debug this problem? I think you will need to add explicit support for junctions.

Thanks,
Gili


#4

I don’t know if Shadow Copy ( https://en.wikipedia.org/wiki/Shadow_Copy ) might be useful here; I’m not a Windows person but I would think about using this to create a snapshot and then rsync the snapshot.

Don’t ask me how though :slight_smile:


#5

Ah! Have you tried with the -L flag?

Yes VSS is the right solution I think. You can create them from powershell I think.

In an idea world rclone would do it… I really need some help with that though so contributions welcome!


#6

Ah! Have you tried with the -L flag?

I was already using this flag. It does not help :frowning:


#7

:frowning:

If you can given instructions on how I can recreate the problem then please make a new issue on github and I’ll take a look.


#8

If you can given instructions on how I can recreate the problem then please make a new issue on github and I’ll take a look.

Filed: https://github.com/ncw/rclone/issues/1973