--filter-from exclude subdirectory in an included directory

Help and Support
1

What is the problem you are having with rclone?

I'm using --filter-from rules to selectively backup directories on my workstation. I have files to include and files to exclude. According to the docs, I should be using --filter-from. However, I need to exclude a subdirectory of parent directory I want to include. I set the exclude first: - Pictures/Photos Library.photoslibrary/" and then the include of the parent: + Pictures/**. But the photoslibary directory keeps getting synced.

Run the command 'rclone version' and share the full output of the command.

❯ rclone version
rclone v1.59.2
- os/version: darwin 12.4 (64 bit)
- os/kernel: 21.5.0 (x86_64)
- os/type: darwin
- os/arch: amd64
- go/version: go1.18.6
- go/linking: dynamic
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync /path/to/home/ remote:home/ \
  --filter-from /path/to/test.filter \
  --tpslimit 10 \
  --vvv \
  --dry-run

The rclone config contents with secrets removed.

NA - not applicable to a remote config.

A log from the command with the -vv flag

...truncated
2022/09/26 09:24:21 NOTICE: Pictures/Photos Library.photoslibrary/resources/derivatives/0/0BA2FB72-20C7-4C1B-B3E5-05F3E962BD23_1_105_c.jpeg: Skipped copy as --dry-run is set (size 160.349Ki)
2022/09/26 09:24:21 NOTICE: Pictures/Photos Library.photoslibrary/resources/derivatives/0/0BBA6939-9A59-49F7-88D3-04867FCA9B7A_1_105_c.jpeg: Skipped copy as --dry-run is set (size 188.662Ki)
2022/09/26 09:24:21 NOTICE: Pictures/Photos Library.photoslibrary/resources/derivatives/0/0B5C0968-CE9C-435E-A14A-5A0A34E4C1B6_1_105_c.jpeg: Skipped copy as --dry-run is set (size 189.823Ki)
...truncated

Copy of the filter

- "Pictures/Photos Libarary.photoslibrary"
+ Pictures/**
- **
2

Can you please share your actual filter file?

3

Try this (note the /** and the end of the first line):

- "Pictures/Photos Libarary.photoslibrary/**"
+ Pictures/**
- **
4

I did right after I posted. Edited the original post.

5

Super - as when I read it, it wasn't there so great that you have it as @Ole already gave you the answer.

6

Still doesn't work. Tries to copy the files.

cc: @Animosity022

7

This is the interactive part - can you share your updated filter and the debug output? You can share use rclone ls to test.

8

I guess I already spotted it:

Libarary -> Library

Tip: copy/paste your filter texts from the log or rclone lsd -R ...

9

Output is too long for a terminal, but when I rclone ls /path/to/home/ with my filter (below), it lists out every file in the photoslibrary folder (which is thousands of files).

rclone ls /path/to/home --filter-from test.filter

> cat test.filter
- "Pictures/Photos Libarary.photoslibrary/**"
+ Pictures/**
- **
10

You have a typo in your first filter line, so it doesn't match anything:

11

Thanks for the typo catch. I fixed it and re-ran with dry run, but it's still listing out files in that directory:

❯ cat test.filter
- "Pictures/Photos Library.photoslibrary/**"
+ Pictures/**
- **
❯ rclone lsd -R /path/to/home --filter-from test.filter | tail
          -1 2022-09-24 17:54:38        -1 Pictures/Photos Library.photoslibrary/resources/derivatives/masters/6
          -1 2022-09-18 02:58:13        -1 Pictures/Photos Library.photoslibrary/resources/derivatives/masters/7
          -1 2022-09-19 16:40:31        -1 Pictures/Photos Library.photoslibrary/resources/derivatives/masters/8
          -1 2022-09-26 08:31:20        -1 Pictures/Photos Library.photoslibrary/resources/derivatives/masters/9
          -1 2022-09-23 20:54:41        -1 Pictures/Photos Library.photoslibrary/resources/derivatives/masters/A
          -1 2022-09-26 08:31:19        -1 Pictures/Photos Library.photoslibrary/resources/derivatives/masters/B
          -1 2022-09-23 15:26:26        -1 Pictures/Photos Library.photoslibrary/resources/derivatives/masters/C
          -1 2022-09-26 08:31:20        -1 Pictures/Photos Library.photoslibrary/resources/derivatives/masters/D
          -1 2022-09-26 08:31:19        -1 Pictures/Photos Library.photoslibrary/resources/derivatives/masters/E
          -1 2022-09-26 08:31:19        -1 Pictures/Photos Library.photoslibrary/resources/derivatives/masters/F
12

Hehe, found something else (I should have seen initially). No quotes in the filter file, unless you expect them to be part of the path or file name. So try this:

- Pictures/Photos Library.photoslibrary/**
+ Pictures/**
- **
13

Hi jiru, in case you missed it, I updated the post above.

14

Ok, that seemed to be the issue. Let me try it again with a dry run.

1 Like
15

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.