--exclude <dir> feature

As documented, RClone only excludes files not directories. Please consider adding an --exclude-dir feature.

Justifications

  1. Users may have a directory of temporary files that they don't need copied to the back-end.
  2. In my case Synology created the #Recycle directory as not readable for mere mortals. From a security perspecive this make sense. This causes an error when trying to copy. RClone developer work-around -- give users read-only access to the directory. That works but not ideal.

Work-around
Clearly one can open up secure directories to copy and then exclude all files but that isn't elegant.

Also, it appears that --exclude will actually silently skip the item. Should there be an option to put a note in the INFO log that files are skipped?

Can you be more specific as you can exclude directories and files.

Examples here:

https://rclone.org/filtering/#patterns-for-matching-path-file-names

From your link:

How filter rules are applied to directories

Rclone commands are applied to path/file names not directories. The entire contents of a directory can be matched to a filter by the pattern directory/* or recursively by directory/**.

In my case, the #Recycle directory isn't readable. RClone then errors out when trying to open the directory that it has no access to. If it has no access it won't be able to get a list of files that exist in the directory for the match.

Per the docs, and RClone developer, I must make the directory at least read-only tor this to work since it will be impossible to open the directory for files. He eluded to others asking for a directory filter.

Make sense?

Can you share an example as an unreadable directory works fine for me on a Linux system:

felix@gemini:~/test$ ls -alR
.:
total 16
drwxrwxr-x  4 felix felix 4096 May 13 21:03 .
drwxr-xr-x 16 felix felix 4096 May 13 10:03 ..
drwx------  2 root  root  4096 May 13 21:02 blah
drwxrwxr-x  2 felix felix 4096 May 13 21:03 blahtwo
ls: cannot open directory './blah': Permission denied

./blahtwo:
total 12
drwxrwxr-x 2 felix felix 4096 May 13 21:03 .
drwxrwxr-x 4 felix felix 4096 May 13 21:03 ..
-rw-r--r-- 1 felix felix  129 May 13 21:03 hosts
felix@gemini:~/test$ rclone ls /home/felix/test
      129 blahtwo/hosts
2021/05/13 21:04:32 ERROR : blah: failed to open directory "blah": open /home/felix/test/blah: permission denied
2021/05/13 21:04:32 Failed to ls: failed to open directory "blah": open /home/felix/test/blah: permission denied
felix@gemini:~/test$ rclone ls /home/felix/test --exclude 'blah/'
      129 blahtwo/hosts
felix@gemini:~/test$

It sure looks like you are correct. Let me put one of my directories on Synology back and see if it behaves correctly.

If correct, i will cross connect the two forum posts to minimize additional confusion.

If correct, should we update the documentation we both referenced?

Allen

I am not sure what's wrong as commands normally work on files and not directories, but filters as noted later apply to directories if you want to use them.

If you have a suggestion for better wording or if something can be clearer, by all means, make a suggestion :slight_smile:

I converted the program config to using your idea and it worked as you expected.

rclone sync --log-level=INFO --exclude "#recycle/" /Volumes/AppleShared Wasabi:LB/AppleShared

Here is the other post: Permissions error on --exclude directory

Regarding this — what about changing docs since it looks like it works as you suggested? Something like include, exclude and filter can match directories or files. A note — this feature can be used to exclude a directory you don’t want that might even have protected non-read access.