Rclone script, filters to back up from Linux to Dropbox with includes, excludes

Help and Support
1

I have the following Linux file structure:

/home/victoria/…
/mnt/Vancouver/…

in which I want to back up selected files and directories to Dropbox.

I have tried includes/excludes files and a filter file, but none are working as intended.

Specifically, I am getting files and directories that I don’t want sync’d, copied over to Dropbox.

For example:

rclone sync --ignore-size --tpslimit 10 /home/victoria/ dropbox:backups/victoria --filter-from /home/victoria/backups/rclone/rclone_filters.txt

# FILTERS (rclone_filters.txt):
+ .adobe/**
+ .avidemux/**
+ .avidemux6/**
+ .mozilla/**
- .cache/**
# EXCLUDE EVERYTHING ELSE:
- *.*

where .adobe/ is ~/.adobe/ , etc. YET ~/.cache/** (etc). are being backed up to Dropbox.

I use / understand rsync (which I use for home backups) – the concept of a snapshot root directory from which paths are defined, etc. – but I find the RClone documentation to be exceedingly confusing.

What am I doing wrong?

2

Your exclude everything rule should be - * since it otherwise allows files without a . to be copied.
The other include rules should work as expected.

rclone filters are a bit tricky to get working exactly as needed. You can use --dump=filters to get the actual set of rules that will be used. This can help to find the correct rules to use.

3

Ahhh, thank you: very much appreciated! I originally had that exclusion (- *) but I was using regex to format my file, and it must have modified it. I’m new to Rclone, so I hadn’t noticed that it had been changed.

Appears to be working now. :slight_smile:

1 Like
4

I have a follow-on question. With the following command,

rclone sync --ignore-size --tpslimit 10 --skip-links  \
/home/victoria/   'dropbox:backups/victoria'  --filter-from \
/home/victoria/backups/rclone/rclone_filters.txt

and this “exclude everything else” [not explicitly included (+) or excluded (-)]

- *

at the end of my filters file, I am still getting folders such as

.gem/
anaconda2/
Desktop/

etc., appearing in Dropbox despite them not being listed (+) in my filters file. Shouldn’t they be excluded by the

- *

flag?

I also get the same result if I explicitly exclude them, e.g.

- .gem/**
- anaconda2/**
- Desktop/**

in that filters file. What’s the issue, here?

5

Are you getting empty folders on dropbox?

Note that the include/exclude rules are processed in order starting from the top so maybe you need to move them higher in the list?

6

No, they are not empty. Thank you for the reminder regarding the order of the rules; I believe that was the problem. Again, I appreciate the assistance. :slight_smile:

https://rclone.org/filtering/#how-the-rules-are-used

Here is a portion of my current filters file. Per your comment I rearranged the order, moving my targeted exclusions (files; folders) to the top of the list.

# EXCLUDED FOLDERS:
- .gem/**
- anaconda2/**
- Desktop/**
- f/**
- file:/**
- pmc_3827080/**

# FILES (HIDDEN LISTED FIRST):
+ .aspell.en.prepl
+ .aspell.en.pws
+ .asunder
+ .babel.json
+ .bash_history
+ .bash_logout
+ .bashrc
#
# ... snip ...
#
+ new
+ new.md
+ new1
+ new2
+ new3
+ new3.md
+ Untitled Document 1
+ Untitled Document 2

# FOLDERS (HIDDEN LISTED FIRST):
+ .adobe/**
+ .audacity-data/**
+ .avidemux/**
+ .avidemux6/**
+ .claws-mail/**
#
# ... snip ...
#
+ .tmux/**
+ .vim/**
+ CytoscapeConfiguration/**
+ linux/**
+ literature/**
+ Mail/**
+ nltk_data/**
+ R_temp/**
+ VirtualBox VMs/**

# EXCLUDE EVERYTHING ELSE:
- *

Update: this has been running for some time (~3+ hours) now, and appears to be fine (my previous problems manifested almost immediately), so I think this “issue” has ben solved. Thanks again! :sun_behind_small_cloud:

1 Like