Ignore all git repos when syncing

Help and Support
1

What is the problem you are having with rclone?

I'm using rclone to make regular offsite backups of my whole Documents folder.

My Documents folder has a lot of git repos in it (I tend to use one-repo-per-project, and this includes all of my projects), and each one has lots of files in it (build artefacts, etc) and that means the backups take a long time.

I don't need my git repos synced, they're all pushed to various remotes as I go. So I want rclone to ignore all git repo folders. I'm currently using an excludes file (i.e. rclone sync--exclude-from ...), but keeping it up-to-date with all my git repos is tedious.

I know that I can exclude files/folders based on globs, but what I'd like to do is have rclone ignore any folder with a .git subfolder. Is this possible?

I know that I could programmatically create the excludes file with something like find ~/Documents -type d -name '.git', but I'm just wondering if there's a way to do that with rclone ignore patterns.

Run the command 'rclone version' and share the full output of the command.

$ rclone --version
rclone v1.59.1
- os/version: darwin 12.6 (64 bit)
- os/kernel: 21.6.0 (arm64)
- os/type: darwin
- os/arch: arm64
- go/version: go1.19
- go/linking: dynamic
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

From my rclone.conf file:

type = webdav
url = https://${OWNCLOUD_URL}/plus/remote.php/webdav/
vendor = owncloud
2

I believe you can use:

Rclone Filtering

and add the .git to that

Test out with dry-run and see if that does what you need or test with rclone ls as that's usually my go to for testing.

3

--exclude ".git/" should do the trick, meaning exclude any directories called .git and their contents.

Try with --dry-run first.

4

I guess I'm confused on the ask.

I have a folder that is a git repo, do you want to exclude just the .git in the repo or whole repo directory?

felix@gemini:/opt/homescripts$ ls -al
total 32
drwxrwxr-x 1 felix felix   132 Aug 11 12:17 .
drwxrwxr-x 1 felix felix   124 Aug 25 16:07 ..
-rw-rw-r-- 1 felix felix  1961 May  7 06:54 Changes.MD
drwxrwxr-x 1 felix felix   204 Aug 11 12:18 .git
-rw-rw-r-- 1 felix felix    18 Apr 14 11:26 .gitignore
-rw-rw-r-- 1 felix felix  4261 Aug 11 12:17 PROXY.MD
-rw-rw-r-- 1 felix felix  1448 May  7 06:54 rclone.conf
-rw-rw-r-- 1 felix felix 11281 Aug 11 12:17 README.md
drwxrwxr-x 1 felix felix   264 May  9 12:29 scripts
drwxrwxr-x 1 felix felix   200 May  7 06:54 systemd

So the .git exclude would work great and remove that folder but not the rest of things.

If you want to remove the whole repo directory, you'd have to use something like exclude if present.

5

Agree, I think @benswift want to exclude all folders containing an immediate child called .git, that is something like

rclone lsd . --exclude-if-present=".git"

except --exclude-if-present only takes a file - not a folder :cry:

Tip: I just placed my repos in a folder called Development and then excluded that, this also fits better for all the test data and debug logs I have floating :sweat_smile:

6

Yep @Ole thats the behaviour I’m after. And you’re also correct that the fact that .git is a folder not a file means that --exclude-if-present isn't a solution either.

Looks like I'll have to go with either the "quarantine all repos into a separate parent folder that doesn't get synced" solution, or the "generate the include file as part of the script" solution I mentioned above.

Thanks anyway.

7

I makes sense to me that exclude if present could be a directory.

If we were worried about backwards compatibility then we could say that if would have to have a trailing / so --exclude-if-present ".git/".

That's probably what I would do. Use find (or rclone) to find all directories called .git then chop the trailing .git off each line and feed to a file and use with --excude-from

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.