Clarification on directory rules and regexps in filters

Help and Support
1

STOP and READ USE THIS TEMPLATE NO EXCEPTIONS - By not using this, you waste your time, our time and really hate puppies. Please remove these two lines and that will confirm you have read them.

What is the problem you are having with rclone?

I’m trying to clarify how rclone’s filter engine behaves when glob rules and regex rules are mixed, because the documentation seems ambiguous.

The docs state that:

Rclone generally accepts Perl-style regular expressions, the exact syntax is defined in the Go regular expression reference. Regular expressions should be enclosed in {{ }}. They will match only the last path segment if the glob doesn't start with / or the whole path name if it does. Note that rclone does not attempt to parse the supplied regular expression, meaning that using any regular expression filter will prevent rclone from using directory filter rules, as it will instead check every path against the supplied regular expression(s).

However, it is not clear whether this affects only traversal performance or also rule semantics and precedence.

Concrete example. Given a tree like:

SUB/a.txt
SUB/inner/b.txt
KEEP/c.txt

And filters:

- /SUB/
+ **

This correctly excludes everything under SUB/.

I’m having trouble understanding the exact semantics of the below filters, esp. w.r.t - /SUB/ when a regexp rule is included, like below:

- /SUB/
+ /{{^SUB/inner/b\.txt$}}
+ **

Questions:

  1. Is filter evaluation still strictly first-match-wins even when regex rules are present?

  2. Is it guaranteed that - /SUB/ will still exclude the entire subtree even if later regex rules would otherwise match files inside it?

In other words: does adding a regex rule alter only performance characteristics, or can it change matching semantics in edge cases?

I have made some experiments with rclone ls that suggest the semantics would be the same in both cases, but I’d like to know for sure.

Run the command 'rclone version' and share the full output of the command.

rclone --version
rclone v1.60.1-DEV
- os/version: ubuntu 24.04 (64 bit)
- os/kernel: 6.17.0-14-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.22.2
- go/linking: dynamic
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

I’m using sftp.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

I made a longer test instructions:

# Create test files
ROOT="$(mktemp -d)"
mkdir -p "$ROOT/SUB/inner" "$ROOT/KEEP"
touch "$ROOT/SUB/a.txt" "$ROOT/SUB/inner/b.txt" "$ROOT/KEEP/c.txt"

find $ROOT
/tmp/tmp.JjhED2Z4J3
/tmp/tmp.JjhED2Z4J3/KEEP
/tmp/tmp.JjhED2Z4J3/KEEP/c.txt
/tmp/tmp.JjhED2Z4J3/SUB
/tmp/tmp.JjhED2Z4J3/SUB/a.txt
/tmp/tmp.JjhED2Z4J3/SUB/inner
/tmp/tmp.JjhED2Z4J3/SUB/inner/b.txt

# Expecting ls to find KEEP/c.txt:
rclone ls "$ROOT" --filter "- /SUB/" --filter "+ **"
        0 KEEP/c.txt

# Expecting ls to NOT find any files from /SUB/ and that's the way it 
# seems to work -- but I can't tell from docs if this is expected!

rclone ls "$ROOT" --filter "- /SUB/" --filter "+ /{{^SUB/inner/b\.txt$}}" --filter "+ **"

        0 KEEP/c.txt

A log from the command that you were trying to run with the -vv flag

$ rclone ls -vv "$ROOT" --filter "- /SUB/" --filter "+ **"
2026/02/25 21:55:39 DEBUG : rclone: Version "v1.60.1-DEV" starting with parameters ["rclone" "ls" "-vv" "/tmp/tmp.JjhED2Z4J3" "--filter" "- /SUB/" "--filter" "+ **"]
2026/02/25 21:55:39 DEBUG : Creating backend with remote "/tmp/tmp.JjhED2Z4J3"
2026/02/25 21:55:39 DEBUG : Using config file from "/home/janne/.config/rclone/rclone.conf"
2026/02/25 21:55:39 DEBUG : SUB: Excluded
        0 KEEP/c.txt
2026/02/25 21:55:39 DEBUG : 2 go routines active

$ rclone ls -vv "$ROOT" --filter "- /SUB/" --filter "+ /{{^SUB/inner/b\.txt$}}" --filter "+ **"
2026/02/25 21:56:12 DEBUG : rclone: Version "v1.60.1-DEV" starting with parameters ["rclone" "ls" "-vv" "/tmp/tmp.JjhED2Z4J3" "--filter" "- /SUB/" "--filter" "+ /{{^SUB/inner/b\\.txt$}}" "--filter" "+ **"]
2026/02/25 21:56:12 DEBUG : Creating backend with remote "/tmp/tmp.JjhED2Z4J3"
2026/02/25 21:56:12 DEBUG : Using config file from "/home/janne/.config/rclone/rclone.conf"
2026/02/25 21:56:12 DEBUG : SUB: Excluded
        0 KEEP/c.txt
2026/02/25 21:56:12 DEBUG : 2 go routines active
2

welcome to the forum,

that is a many years old, custom compiled, development version of rclone from an out of date repository.
please rclone selfupdate and test again.

3

Filters are top down matching so you want your tightest filters at the top and the less ones at the bottom.

So that first - is going to caus ethe + below it to never match - flip the order.

4

Thanks for the replies @asdffdsa and @Animosity022! Actually the tool is working correctly and logically to me, but I’m asking for clarification on the docs, because the documentation (for latest versions) suggests that the - /SUB/ might get affected by further regexp filter rules. That doesn’t seem to be the case in my testing (examples in my original post), but I’d like to confirm this for sure, so that production systems do not break when the filter rules are slightly modified or new (ones including regexps) filter rules get added.

5

I dont’ see that in the docs anywhere as I shared that it’s top down so if you drop a directory path, nothing below it will pick it up again.

Also, many things have changed in rclone over the years and the docs update with the version. Generally, there aren’t major breaking changes but testing on a mega old version and comparing it to the docs might not be great as things get fixed/changed.

This area speaks to it:

Rclone Filtering

By changing the order:

felix@neurotic:~/test$ rclone ls "$ROOT" --filter "+ /{{^SUB/inner/b.txt$}}" --filter "- /SUB/" --filter "+ **"
0 SUB/inner/b.txt
0 KEEP/c.txt

1 Like
6

Thanks! The part in the docs that confuses me (from your linked page) is this:

Rclone generally accepts Perl-style regular expressions, the exact syntax is defined in the Go regular expression reference. Regular expressions should be enclosed in {{}}. They will match only the last path segment if the glob doesn't start with / or the whole path name if it does. Note that rclone does not attempt to parse the supplied regular expression, meaning that using any regular expression filter will prevent rclone from using directory filter rules, as it will instead check every path against the supplied regular expression(s).

(emphasis mine)

My worry is that - /SUB/ would get disabled if the rule list contains any filter rules with regular expressions.

Your reply seems to indicate that is not the case, and my experiments with rclone indicate the same. The logical thing in the presence of regular expressions would be for the filter engine to treat - /SUB/ as a regular expression too but the docs instead say regular expression filter will prevent rclone from using directory filter rules.