--filter-from not working

What is the problem you are having with rclone?

--filter-from is not working

filter.txt

- desktop.ini
+ /Desktop/**
+ /Downloads/**
+ /Documents/**
+ /Pictures/**
+ /Videos/**
+ /AppData/Local/Google/Chrome/**

Run the command 'rclone version' and share the full output of the command.

rclone v1.64.0
  • os/version: Microsoft Windows 10 Pro 22H2 (64 bit)
  • os/kernel: 10.0.19045.3448 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.21.1
  • go/linking: static
  • go/tags: cmount

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone lsd c:/users/af --filter-from "C:/Users/AF/filter.txt"

A log from the command that you were trying to run with the -vv flag

2023/09/23 12:29:55 DEBUG : rclone: Version "v1.64.0" starting with parameters ["C:\\rclone\\rclone.exe" "lsd" "C:/Users/AF" "--filter-from" "C:/Users/AF/filter.txt" "-vv"]
2023/09/23 12:29:55 DEBUG : Creating backend with remote "C:/Users/AF"
2023/09/23 12:29:55 DEBUG : Using config file from "C:\\Users\\AF\\AppData\\Roaming\\rclone\\rclone.conf"
2023/09/23 12:29:55 DEBUG : fs cache: renaming cache item "C:/Users/AF" to be canonical "//?/C:/Users/AF"
2023/09/23 12:29:55 NOTICE: Application Data: Can't follow symlink without -L/--copy-links
2023/09/23 12:29:55 NOTICE: Cookies: Can't follow symlink without -L/--copy-links
2023/09/23 12:29:55 NOTICE: Local Settings: Can't follow symlink without -L/--copy-links
2023/09/23 12:29:55 NOTICE: My Documents: Can't follow symlink without -L/--copy-links
2023/09/23 12:29:55 NOTICE: NetHood: Can't follow symlink without -L/--copy-links
2023/09/23 12:29:55 NOTICE: PrintHood: Can't follow symlink without -L/--copy-links
2023/09/23 12:29:55 NOTICE: Recent: Can't follow symlink without -L/--copy-links
2023/09/23 12:29:55 NOTICE: SendTo: Can't follow symlink without -L/--copy-links
2023/09/23 12:29:55 NOTICE: Start Menu: Can't follow symlink without -L/--copy-links
2023/09/23 12:29:55 NOTICE: Templates: Can't follow symlink without -L/--copy-links
          -1 2023-08-08 12:11:07        -1 .1password
          -1 2023-05-22 22:18:42        -1 .android
          -1 2023-02-26 01:04:32        -1 .cache
          -1 2023-04-29 16:39:08        -1 .config
          -1 2023-09-21 11:50:43        -1 .dbus-keyrings
          -1 2023-05-22 22:25:10        -1 .fleet
          -1 2023-07-03 21:22:50        -1 .vscode
          -1 2023-02-22 12:13:51        -1 3D Objects
          -1 2023-02-22 12:13:36        -1 AppData
          -1 2023-02-22 12:13:51        -1 Contacts
          -1 2023-09-23 04:49:37        -1 Desktop
          -1 2023-09-22 17:47:22        -1 Documents
          -1 2023-09-23 04:48:39        -1 Downloads
          -1 2023-02-22 12:13:51        -1 Favorites
          -1 2023-09-21 09:43:29        -1 IntelGraphicsProfiles
          -1 2023-02-22 12:13:51        -1 Links
          -1 2023-06-16 16:01:42        -1 Music
          -1 2023-09-22 18:25:06        -1 Pictures
          -1 2023-02-22 12:13:51        -1 Saved Games
          -1 2023-02-22 12:15:01        -1 Searches
          -1 2023-09-11 10:49:08        -1 Videos
2023/09/23 12:29:55 DEBUG : 2 go routines active

It seems to be working perfectly fine.... It does exactly what you specified.
Why you think it is not? Or what is what you want to achieve?

You can check if your filters are applied or not by adding --dump filters flag to your command.

Shouldn't Rclone only list/copy directories with + and ignore everything else, similar to --include ?

PS C:\Users\AF> rclone copy c:/users/af db:af --tpslimit 12 --filter-from "c:/users/af/filter.txt" --dump filters
2023/09/23 14:44:40 NOTICE: Automatically setting -vv as --dump is enabled
2023/09/23 14:44:40 INFO  : Starting transaction limiter: max 12 transactions/s with burst 1
--- start filters ---
--- File filter rules ---
- (^|/)desktop\.ini$
+ ^Desktop/.*$
+ ^Downloads/.*$
+ ^Documents/.*$
+ ^Pictures/.*$
+ ^Videos/.*$
+ ^AppData/Local/Google/Chrome/.*$
--- Directory filter rules ---
+ ^Desktop/.*/$
+ ^Desktop/$
+ ^Desktop/.*$
+ ^Downloads/.*/$
+ ^Downloads/$
+ ^Downloads/.*$
+ ^Documents/.*/$
+ ^Documents/$
+ ^Documents/.*$
+ ^Pictures/.*/$
+ ^Pictures/$
+ ^Pictures/.*$
+ ^Videos/.*/$
+ ^Videos/$
+ ^Videos/.*$
+ ^AppData/Local/Google/Chrome/.*/$
+ ^AppData/Local/Google/Chrome/$
+ ^AppData/Local/Google/$
+ ^AppData/Local/$
+ ^AppData/$
+ ^AppData/Local/Google/Chrome/.*$
--- end filters ---
2023/09/23 14:44:40 DEBUG : rclone: Version "v1.64.0" starting with parameters ["C:\\rclone\\rclone.exe" "copy" "c:/users/af" "db:af" "--tpslimit" "12" "--filter-from" "c:/users/af/filter.txt" "--dump" "filters"]
2023/09/23 14:44:40 DEBUG : Creating backend with remote "c:/users/af"
2023/09/23 14:44:40 DEBUG : Using config file from "C:\\Users\\AF\\AppData\\Roaming\\rclone\\rclone.conf"
2023/09/23 14:44:40 DEBUG : fs cache: renaming cache item "c:/users/af" to be canonical "//?/c:/users/af"
2023/09/23 14:44:40 DEBUG : Creating backend with remote "db:af"
2023/09/23 14:44:40 DEBUG : Creating backend with remote "dropbox:荀
2023/09/23 14:44:42 DEBUG : .esd_auth: Size and modification time the same (differ by -316.1548ms, within tolerance 1s)
2023/09/23 14:44:42 DEBUG : downloader.log: Size and modification time the same (differ by -328.752ms, within tolerance 1s)
2023/09/23 14:44:42 DEBUG : downloader.log: Unchanged skipping
2023/09/23 14:44:42 DEBUG : filter.txt: Size and modification time the same (differ by 54.5602ms, within tolerance 1s)
2023/09/23 14:44:42 DEBUG : filter.txt: Unchanged skipping
2023/09/23 14:44:42 DEBUG : ntuser.ini: Size and modification time the same (differ by -317.6251ms, within tolerance 1s)2023/09/23 14:44:42 DEBUG : ntuser.ini: Unchanged skipping
2023/09/23 14:44:42 DEBUG : .esd_auth: Unchanged skipping
2023/09/23 14:44:42 DEBUG : NTUSER.DAT{53b39e88-18c4-11ea-a811-000d3aa4692b}.TM.blf: Size and modification time the same (differ by 391.1385ms, within tolerance 1s)
2023/09/23 14:44:42 DEBUG : NTUSER.DAT{53b39e88-18c4-11ea-a811-000d3aa4692b}.TM.blf: Unchanged skipping
2023/09/23 14:44:42 ERROR : ntuser.dat.LOG1: Failed to copy: failed to open source object: The process cannot access the file because it is being used by another process.
2023/09/23 14:44:42 DEBUG : NTUSER.DAT{53b39e88-18c4-11ea-a811-000d3aa4692b}.TMContainer00000000000000000002.regtrans-ms: Size and modification time the same (differ by -270.7578ms, within tolerance 1s)
2023/09/23 14:44:42 DEBUG : NTUSER.DAT{53b39e88-18c4-11ea-a811-000d3aa4692b}.TMContainer00000000000000000002.regtrans-ms: Unchanged skipping
2023/09/23 14:44:42 DEBUG : URLs.txt: Size and modification time the same (differ by -323.5558ms, within tolerance 1s)
2023/09/23 14:44:42 DEBUG : config.yaml: Size and modification time the same (differ by -259.8734ms, within tolerance 1s)
2023/09/23 14:44:42 DEBUG : .gtk-bookmarks: Size and modification time the same (differ by -116.4236ms, within tolerance 1s)
2023/09/23 14:44:42 ERROR : NTUSER.DAT: Failed to copy: failed to open source object: The process cannot access the file because it is being used by another process.
2023/09/23 14:44:42 ERROR : ntuser.dat.LOG2: Failed to copy: failed to open source object: The process cannot access the file because it is being used by another process.
2023/09/23 14:44:42 NOTICE: Documents/My Music: Can't follow symlink without -L/--copy-links
2023/09/23 14:44:42 DEBUG : NTUSER.DAT{53b39e88-18c4-11ea-a811-000d3aa4692b}.TMContainer00000000000000000001.regtrans-ms: Size and modification time the same (differ by -270.7578ms, within tolerance 1s)
2023/09/23 14:44:42 DEBUG : NTUSER.DAT{53b39e88-18c4-11ea-a811-000d3aa4692b}.TMContainer00000000000000000001.regtrans-ms: Unchanged skipping
2023/09/23 14:44:42 DEBUG : URLs.txt: Unchanged skipping
2023/09/23 14:44:42 DEBUG : config.yaml: Unchanged skipping
2023/09/23 14:44:42 DEBUG : .gtk-bookmarks: Unchanged skipping
2023/09/23 14:44:42 NOTICE: Documents/My Pictures: Can't follow symlink without -L/--copy-links
2023/09/23 14:44:42 NOTICE: Documents/My Videos: Can't follow symlink without -L/--copy-links
2023/09/23 14:44:42 DEBUG : .fleet/settings.json: Size and modification time the same (differ by 412.6133ms, within tolerance 1s)
2023/09/23 14:44:42 DEBUG : .fleet/settings.json: Unchanged skipping
2023/09/23 14:44:43 DEBUG : .dbus-keyrings/org_gtk_gdbus_general: Size and modification time the same (differ by -186.2621ms, within tolerance 1s)
2023/09/23 14:44:43 DEBUG : .dbus-keyrings/org_gtk_gdbus_general: Unchanged skipping
2023/09/23 14:44:43 DEBUG : .vscode/argv.json: Size and modification time the same (differ by 432.473ms, within tolerance 1s)
2023/09/23 14:44:43 DEBUG : .vscode/argv.json: Unchanged skipping

OK now clear. As per docs you need the following rules:

# exclude file does not matter where it is
- desktop.ini
# include specific folders
+ /Desktop/**
+ /Downloads/**
+ /Documents/**
+ /Pictures/**
+ /Videos/**
+ /AppData/Local/Google/Chrome/**
# exclude everything else
- *

When rules from --filter-from are applied they are evaluated one by one until there is a match. Without - * at the end no rule is applied for e.g. /Test and it is included as well.

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.