Rclone filter-from file

What is the problem you are having with rclone?

I have a filter file, where I want to INCLUDE, for example, /appdata/radarr/backups/, but EXCLUDE every other file and subdirectory in /appdata/radarr/.

My understanding after reading this thread was that filter files are read from top down, so I set up my filter file like below but /appdata/radarr/backups/ is still excluded (see log).

# Exclude general folders
- /data/**
- /domains/**
- /isos/**
- /system/**
- /appdata/plex/Library/Application Support/Plex Media Server/Cache/**
- /appdata/plex/Library/Application Support/Plex Media Server/Media/**
- /appdata/plex/Library/Application Support/Plex Media Server/Metadata/**
- /appdata/plex/Library/Application Support/Plex Media Server/Logs/**
- /appdata/plex/Library/Application Support/Plex Media Server/Crash Reports/**
- /appdata/other/**
- /appdata/jackett/**
- /appdata/QDirStat/**

# Include specific backup folders
+ /appdata/radarr/backups/**
- /appdata/radarr/**
+ /appdata/sonarr/backups/**
- /appdata/sonarr/**
+ /appdata/lidarr/backups/**
- /appdata/lidarr/**

# Exclude specific hidden and temporary files
- *fuse_hidden*
- *_HIDDEN
- .recycle**
- .Recycle.Bin/**
- *.backup~*
- *.partial~*

Run the command 'rclone version' and share the full output of the command.

rclone v1.69.0

  • os/version: slackware 15.0+ (64 bit)
  • os/kernel: 6.6.68-Unraid (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.23.4
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

I'm running rclone in a script:

2025/02/08 12:21:21 DEBUG : rclone: Version "v1.69.0" starting with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone/.rclone.conf" "copy" "/mnt/user/" "gd:backup" "--backup-dir" "gd:backup-deleted" "--user-agent=gd" "-vv" "--buffer-size" "512M" "--drive-chunk-size" "512M" "--tpslimit" "8" "--checkers" "8" "--transfers" "4" "--order-by" "modtime,ascending" "--min-age" "15m" "--filter-from" "/mnt/user/appdata/other/rclone/remotes/gd/rclone_filters.txt" "--drive-stop-on-upload-limit" "--bwlimit" "01:00,off 08:00,15M 16:00,12M" "--bind="]

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[gd]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
root_folder_id = XXX
server_side_across_configs = true
team_drive =

A log from the command that you were trying to run with the -vv flag

08.02.2025 12:21:21 Creating filter file...
08.02.2025 12:21:21 Filter file created at /mnt/user/appdata/other/rclone/remotes/gd/rclone_filters.txt
2025/02/08 12:21:21 DEBUG : rclone: Version "v1.69.0" starting with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone/.rclone.conf" "copy" "/mnt/user/" "gd:backup" "--backup-dir" "gd:backup-deleted" "--user-agent=gd" "-vv" "--buffer-size" "512M" "--drive-chunk-size" "512M" "--tpslimit" "8" "--checkers" "8" "--transfers" "4" "--order-by" "modtime,ascending" "--min-age" "15m" "--filter-from" "/mnt/user/appdata/other/rclone/remotes/gd/rclone_filters.txt" "--drive-stop-on-upload-limit" "--bwlimit" "01:00,off 08:00,15M 16:00,12M" "--bind="]
2025/02/08 12:21:21 DEBUG : Creating backend with remote "/mnt/user/"
2025/02/08 12:21:21 DEBUG : Using config file from "/boot/config/plugins/rclone/.rclone.conf"
2025/02/08 12:21:21 DEBUG : fs cache: renaming cache item "/mnt/user/" to be canonical "/mnt/user"
2025/02/08 12:21:21 DEBUG : data: Excluded
2025/02/08 12:21:21 DEBUG : isos: Excluded
2025/02/08 12:21:21 DEBUG : system: Excluded
2025/02/08 12:21:21 DEBUG : domains: Excluded
2025/02/08 12:21:22 DEBUG : appdata: Directory modification time the same (differ by -505.105µs, within tolerance 1ms)
2025/02/08 12:21:22 DEBUG : Added delayed dir = "appdata", newDst=appdata
2025/02/08 12:21:22 DEBUG : appdata/other: Excluded
2025/02/08 12:21:22 DEBUG : appdata/jackett: Excluded
2025/02/08 12:21:22 DEBUG : appdata/QDirStat: Excluded
2025/02/08 12:21:22 DEBUG : Added delayed dir = "appdata/cloudflared", newDst=<nil>
2025/02/08 12:21:22 DEBUG : Added delayed dir = "appdata/lidarr", newDst=<nil>
2025/02/08 12:21:22 DEBUG : Added delayed dir = "appdata/radarr", newDst=<nil>
2025/02/08 12:21:22 DEBUG : Added delayed dir = "appdata/sonarr", newDst=<nil>
2025/02/08 12:21:22 DEBUG : appdata/arcbrowser: Directory modification time the same (differ by -905.783µs, within tolerance 1ms)
2025/02/08 12:21:22 DEBUG : Added delayed dir = "appdata/arcbrowser", newDst=appdata/arcbrowser
2025/02/08 12:21:22 DEBUG : appdata/plex: Directory modification time the same (differ by -231.815µs, within tolerance 1ms)
2025/02/08 12:21:22 DEBUG : Added delayed dir = "appdata/plex", newDst=appdata/plex
2025/02/08 12:21:22 DEBUG : appdata/retroarch: Directory modification time the same (differ by -905.756µs, within tolerance 1ms)
2025/02/08 12:21:22 DEBUG : Added delayed dir = "appdata/retroarch", newDst=appdata/retroarch
2025/02/08 12:21:22 DEBUG : appdata/transmission: Directory modification time the same (differ by -702.841µs, within tolerance 1ms)
2025/02/08 12:21:22 DEBUG : Added delayed dir = "appdata/transmission", newDst=appdata/transmission
2025/02/08 12:21:22 DEBUG : appdata/lidarr/logs: Excluded
2025/02/08 12:21:22 DEBUG : appdata/lidarr/asp: Excluded
2025/02/08 12:21:22 DEBUG : appdata/lidarr/MediaCover: Excluded
2025/02/08 12:21:22 DEBUG : appdata/lidarr/Backups: Excluded
2025/02/08 12:21:22 DEBUG : appdata/radarr/logs: Excluded
2025/02/08 12:21:22 DEBUG : appdata/radarr/asp: Excluded
2025/02/08 12:21:22 DEBUG : appdata/radarr/MediaCover: Excluded
2025/02/08 12:21:22 DEBUG : appdata/radarr/Backups: Excluded
2025/02/08 12:21:22 DEBUG : appdata/radarr/Sentry: Excluded
2025/02/08 12:21:22 DEBUG : appdata/sonarr/logs: Excluded
2025/02/08 12:21:22 DEBUG : appdata/sonarr/.config: Excluded
2025/02/08 12:21:22 DEBUG : appdata/sonarr/MediaCover: Excluded
2025/02/08 12:21:22 DEBUG : appdata/sonarr/Backups: Excluded
2025/02/08 12:21:22 DEBUG : appdata/sonarr/asp: Excluded
2025/02/08 12:21:22 DEBUG : appdata/sonarr/Sentry: Excluded

Not sure what your question really is...

but if you wonder why below is excluded

then the reason is that filters are case sensitive and your include filter applies to backups and not Backups

Otherwise your understanding of filters is 100% correct:)

If you do not want to worry about case sensitivity add --ignore-case flag to your command.

If my understanding of your issue is wrong then feel free to clarify.

Figured it was something stupid! Thanks!

1 Like

Filters are surprisingly hard to get them right.

Good idea is to use rclone ls for testing and add --dump filters for harder cases - to see what filters are really applied by rclone. Latter maybe more useful when using regex etc.

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.