I made an rclone filters generator

Demo:
rclone genfilters demo

Works with any rclone remote, and at the end it spits out a file that can be used as a --filter-from.

I'm still working on it but was excited about it and thought I'd share :slight_smile: It is written in Go and very much inspired by rclone's ncdu and tree. It supports both - ** and + ** modes, and can also dump filters as regex, and it is careful about the rule order and avoiding redundancy. It all runs through rclone's actual filter module in realtime, so you can see exactly what you're getting.

7 Likes

Could you release alpha/beta to play with? It looks as a potentially very handy addition to rclone users.

1 Like

Sure, here you go:

If you try it out and have any feedback, feel free to let me know :slightly_smiling_face:

2 Likes

nifty tool, thanks!

1 bad decision though: You expect rclone to be 'installed'/configured on the target system. Write your output file rather in the current folder rclone is run from.

2023/12/17 13:54:13 NOTICE: Example usage:
rclone tree "C:\" --filter-from "C:\Users\[user]\AppData\Local\rclone\rclone_genfilters.txt"
2023/12/17 13:54:13 ERROR : C:\Users[user]\AppData\Local\rclone\rclone_genfilters.txt: error writing file: open C:\Users[user]\AppData\Local\rclone\rclone_genfilters.txt: The system cannot find the path specified.
2023/12/17 13:54:13 ERROR : C:\Users[user]\AppData\Local\rclone\rclone_genfilters.txt: error writing file: invalid argument

Thanks for the feedback! You can customize the path for the output file with the -o flag, like:

rclone genfilters remote:path -o /some/custom/path.txt

If no path is specified, it currently defaults to rclone's default cache dir. Is there a different dir that would be a better default?

Thanks for replying!

Is there a different dir that would be a better default?

IMO yes, as I said: Follow Win/*nix standards - the current dir if no other is specified.

(Also I dislike 3rd party tools writing to other apps config(dirs), IMO a bad practice.
But maybe @ncw likes to integrate your work into standard rclone?).

1 Like

another possible improvement:
Allow (use) wider screens, (allow to) put the tree more to the right so the filter text isn't cut after a few characters.

edit:
Dunno if a program can read screen parameters but 50% of the width appears a pretty good place as standard to place the tree so tree and filter have the same space to display looong names.

1 Like

Good suggestions, thank you! I will take a look at that.

This is a nice tool :slightly_smiling_face:

It could potentially be an rclone sub command?

2 Likes

I would be happy to submit a PR :slightly_smiling_face:

@rcRsc just wanted to let you know I have posted an updated version that implements both of your suggestions. Thanks again for the feedback :slight_smile:

2 Likes

Cool, thank you!
:open_mouth: But where did you get rclone 1.66 from?

@ncw apparently rclone 1.66 (at least @nielash​'s version) solved the open bug:

It is based on a beta version v1.66.0-beta.7579.743ea6ac2, available here: https://beta.rclone.org/v1.66.0-beta.7579.743ea6ac2/ (latest is here.)

I think the bug you're referring to was a bug in go1.21.4 (not technically a bug in rclone.) The genfilters binaries were compiled with go1.21.5, which probably explains why it is fixed. So I can't take credit :slightly_smiling_face: It is probably also fixed in the latest official beta of rclone.

Technically it is a bug in rclone.
I've been a programmer too 'in my former life', the components I decided to use (to save much work) were my responsibility.

Did Nick do anything wrong? No, not at all! Still the software failed, so it had a bug.

This fix is being released in 1.65.1 which should be out tomorrow probably :slight_smile:

1 Like

Generated filters.txt by genfilters and tried mounting with --filter-from=filters.txt, there seems a bug cannot handling square brackets. Am I the only one experiencing this problem?

I think the issue is probably that square brackets in filter patterns are interpreted as a character-range, so they need to be escaped with \ if you want them interpreted literally.

For example, instead of:

- /filewith[brackets]inname

try:

- /filewith\[brackets\]inname

But I agree genfilters should probably be smarter about detecting this and escaping automatically! Thank you for bringing this to my attention -- I have some revisions planned already and I will try to include this :slightly_smiling_face:

1 Like