Exclusion filter not working on specific directory, works everywhere else

What is the problem you are having with rclone?

I'm ultimately working to copy multiple directories while using exclusion filters to skip several. I'm using lsd to test my filters.

I can use --exclude & --exclude-from file.txt successfully on every directory except for one specific directory within a specific directory

rclone lsd D:\Home --exclude "Brunch@Bobby's/**"

The directory "Brunch@Bobby's/**" is not excluded.

As a test, I created a directory by the same name in another location on the server. Here the exclusion filter works as expected and is successful.

reclone lsd D:\test --exclude "Brunch@Bobby's/**"

The issue is limited to the directory D:\Home\Brunch@Bobby's ... I'm unable to get this single directory to be excluded. My Windows User has full rights on this directory and seems to have the same permissions as every other directory.

Any tips or tricks for anyone coming across this issue before?

What is your rclone version (output from rclone version)

v1.53.0

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows Server 2012 R2 Standard, 64 bit

Which cloud storage system are you using? (eg Google Drive)

Ultimately G Suite Shared Drives (previously Team Drives)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone lsd D:\Home --exclude "Brunch@Bobby's/**"

The rclone config contents with secrets removed.

[baker-pilot]
type = drive
scope = drive
token = {"access_token":"redacted","token_type":"Bearer","refresh_token":"redacted","expiry":"2020-09-23T05:15:51.58149-04:00"}
team_drive = redacted
root_folder_id = 

[homedrive]
type = drive
scope = drive
token = {"access_token":"redacted","token_type":"Bearer","refresh_token":"redacted","expiry":"2020-09-23T13:42:39.8142637-04:00"}
team_drive = redacted
root_folder_id = 

A log from the command with the -vv flag

C:\rclone>rclone lsd D:\Home --exclude "Brunch@Bobby's/" -vv --dump filters
--- start filters ---
--- File filter rules ---
--- Directory filter rules ---
- (^|/)Brunch@Bobby's/$
--- end filters ---
2020/09/24 11:39:09 DEBUG : rclone: Version "v1.53.0" starting with parameters [
"rclone" "lsd" "D:\\Home" "--exclude" "Brunch@Bobby's/" "-vv" "--dump" "filters"
]
2020/09/24 11:39:09 DEBUG : Creating backend with remote "D:\\Home"
2020/09/24 11:39:09 DEBUG : Using config file from "C:\\Users\\googadmin\\.confi
g\\rclone\\rclone.conf"
2020/09/24 11:39:09 DEBUG : fs cache: renaming cache item "D:\\Home" to be canon
ical "//?/D:/Home"
          -1 2018-07-16 17:34:33        -1 .TemporaryItems
          -1 2018-07-16 17:34:33        -1 .Trashes
          -1 2018-07-16 17:34:33        -1 redacted
          -1 2015-02-04 19:15:13        -1 AE
          -1 2020-08-09 22:34:23        -1 AV Defender
          -1 2020-03-24 14:53:38        -1 Accounting
          -1 2017-08-25 16:26:56        -1 Act Backup
          -1 2018-03-13 10:07:30        -1 redacted
          -1 2016-10-05 13:34:43        -1 redacted
          -1 2018-10-26 16:41:06        -1 BBPost-BDev
          -1 2018-07-26 16:51:22        -1 redacted
          -1 2020-02-11 11:21:55        -1 redacted
          -1 2017-10-20 12:44:54        -1 redacted
          -1 2020-04-03 16:20:57        -1 redacted
          -1 2020-02-04 13:17:42        -1 redacted
          -1 2017-12-04 11:51:01        -1 redacted
          -1 2010-05-20 13:28:27        -1 redacted
          -1 2018-06-02 02:03:57        -1 redacted
          -1 2019-06-17 13:46:58        -1 redacted
          -1 2018-01-31 11:22:29        -1 Brunch@Bobby's
          -1 2008-07-17 12:29:04        -1 Budgets
          -1 2016-09-23 15:51:18        -1 Casting
          -1 2017-09-14 18:04:10        -1 City of East hampton
          -1 2020-09-21 18:09:08        -1 redacted
          -1 2020-08-12 16:34:51        -1 DEVELOPMENT
          -1 2016-01-11 18:23:23        -1 Dev-Legal
          
2020/09/24 11:39:09 DEBUG : 2 go routines active

You just need the ** after the exclude:

felix@guardian:~/test$ rclone ls /home/felix/test
1504953150 jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv
        0 Brunch@Bobby's/blah
        0 .test/blah
felix@guardian:~/test$ rclone ls /home/felix/test --exclude "Brunch@Bobby's/**"
1504953150 jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv
        0 .test/blah

I'm able to use that cmd to filter from any other directory except for /home

rclone lsd /home --exclude "Brunch@Bobby's/**" 

...does not work.

rclone lsd/test --exclude "Brunch@Bobby's/**" 

...does work.

2020/09/24 14:20:01 DEBUG : rclone: Version "v1.53.0" starting with parameters [
"rclone" "lsd" "D:\\test" "--exclude" "Brunch@Bobby's/**" "-vv"]
2020/09/24 14:20:01 DEBUG : Creating backend with remote "D:\\test"
2020/09/24 14:20:01 DEBUG : Using config file from "C:\\Users\\googadmin\\.confi
g\\rclone\\rclone.conf"
2020/09/24 14:20:01 DEBUG : fs cache: renaming cache item "D:\\test" to be canon
ical "//?/D:/test"
2020/09/24 14:20:01 DEBUG : Brunch@Bobby's: Excluded
          -1 2017-02-14 20:17:50        -1 BeatBF
          -1 2019-09-17 13:13:45        -1 James
          -1 2019-09-17 12:44:53        -1 Tiara
2020/09/24 14:20:01 DEBUG : 2 go routines active

C:\rclone>rclone lsd D:\Home --exclude "Brunch@Bobby's/**" -vv
2020/09/24 14:20:06 DEBUG : rclone: Version "v1.53.0" starting with parameters [
"rclone" "lsd" "D:\\Home" "--exclude" "Brunch@Bobby's/**" "-vv"]
2020/09/24 14:20:06 DEBUG : Creating backend with remote "D:\\Home"
2020/09/24 14:20:06 DEBUG : Using config file from "C:\\Users\\googadmin\\.confi
g\\rclone\\rclone.conf"
2020/09/24 14:20:06 DEBUG : fs cache: renaming cache item "D:\\Home" to be canon
ical "//?/D:/Home"
          -1 2018-07-16 17:34:33        -1 .TemporaryItems
          -1 2018-07-16 17:34:33        -1 .Trashes
          -1 2018-07-16 17:34:33        -1 5IngredientsFix-unused
          -1 2015-02-04 19:15:13        -1 AE
          -1 2020-08-09 22:34:23        -1 AV Defender
          -1 2020-03-24 14:53:38        -1 Accounting
          -1 2017-08-25 16:26:56        -1 Act Backup
          -1 2018-03-13 10:07:30        -1 Alex's Day Off
          -1 2016-10-05 13:34:43        -1 BB Website 9-21-2016
          -1 2018-10-26 16:41:06        -1 BBPost-BDev
          -1 2018-07-26 16:51:22        -1 BBPost_BDev_2018
          -1 2020-02-11 11:21:55        -1 BBQ Brawl
          -1 2017-10-20 12:44:54        -1 BRUNCH 7 Acct Dept
          -1 2020-04-03 16:20:57        -1 BeatBF
          -1 2020-02-04 13:17:42        -1 Bobby + Giada Italy
          -1 2017-12-04 11:51:01        -1 Bobby and Damaris Show
          -1 2010-05-20 13:28:27        -1 BobbyFlay
          -1 2018-06-02 02:03:57        -1 Bread & Butter
          -1 2019-06-17 13:46:58        -1 Breeders' Cup
          -1 2018-01-31 11:22:29        -1 Brunch@Bobby's
          -1 2008-07-17 12:29:04        -1 Budgets

To progress, you'd have to share those outputs with debug as we can't see what's going on.

I added some additional info. I'm assuming it has to be some sort of attribute of the directory.

Confirmed it's some sort of permissions issue on the directory ... I'm not able to run any rclone cmds against D:/Home/Brunch@Bobby's

...I am going to go review the forums for directory permissions troubleshooting.

Yep, rclone needs permissions to exclude something as it's unable to check that directory.

I was going to say to run a

rclone ls "D:\\Home\Brunch@Bobby's" -vv

and that should show the permission error.

1 Like

rclone would state the directory was not found.

I ran a permissions check in Windows and found: "No mapping between account names and security IDs was done"

It looks like I was able to fix it simply by moving the data, recreating the directory, and copying the directories back into the new D:/Home/Brunch@Bobby's directory.

This fix works b/c they're not actively working in this directory. The 'real' fix requires admin rights to the Windows domain controller which I don't think I currently have.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.