Excluding files/directories with remote control api

Help and Support
1

What is the problem you are having with rclone?

I control rclone remotely with the api. I want to exclude some directories by using the ExcludeFrom filter.

I've set the filter to refer to a file. In the file I've listed test files/directories several ways trying to determine proper syntax. This file path path is in the rclone jail.

When I start the job and watch the logs, the excluded files are still copied. Backups work fine, just not exclusion.

I've read several posts about excluding, rc, and I think I have the right syntax. I even tried excluding a single file like home-assistant.log, setting in in ExlcudeFile but it was still processed.

I have rlcone rcd logging at debug and the main loglevel at 10.

What is your rclone version (output from rclone version)

root@rclone:~ # rclone version
rclone v1.51.0

  • os/arch: freebsd/amd64
  • go version: go1.14.7

Which OS you are using and how many bits (eg Windows 7, 64 bit)

FreeNAS 11.2, rclone is in a 11.1 jail.

Which cloud storage system are you using? (eg Google Drive)

b2

The command you were trying to run (eg rclone copy /tmp remote:tmp)

/root/rclone_exclude: (these are relative to the src specified in the api post body)

backuptest
home-assistant.log
tts/**

The rclone config contents with secrets removed.

rc options/get:

{
        "filter": {
                "DeleteExcluded": false,
                "ExcludeFile": "",
                "ExcludeFrom": [
                        "/root/rclone_exclude"
                ],
                "ExcludeRule": null,
                "FilesFrom": null,
                "FilterFrom": null,
                "FilterRule": null,
                "IgnoreCase": false,
                "IncludeFrom": null,
                "IncludeRule": null,
                "MaxAge": 9223372036854776000,
                "MaxSize": -1,
                "MinAge": 9223372036854776000,
                "MinSize": -1
        },

rclone.conf:

[remoteb2]
type = b2
account = <account>
key = <key>
endpoint = 

[remoteb2-encrypt]
type = crypt
remote = remoteb2:<remote>
filename_encryption = obfuscate
password = <pass>
password2 = <pass2>

A log from the command with the -vv flag

I grepped the running log from the rc backup call for an excluded file

curl /sync/sync -H "Content-Type: application/json" -d {{"min-age": 15, "transfers": 16, "_async": "true", "srcFs": /mnt/pool1/main/nfs/hass, "dstFs": remoteb2-encrypt:/main/nfs/hass'}

2020/08/17 11:48:43 DEBUG : home-assistant.log: Sizes differ (src 2957824 vs dst 2937658)
2020/08/17 11:48:47 INFO  : home-assistant.log: Copied (replaced existing)
2

Run the rclone rcd with --dump filters and quit it after your attempted copy. It will show you what it thinks the filters are.

3

Thank you for the suggestion

I started rcd with the current command adding --dump filters, command and output below

root@rclone:~ # /usr/local/bin/rclone rcd --fast-list --dump filters --rc-no-auth --log-file /var/log/rclone.log --log-level DEBUG
--- start filters ---
--- File filter rules ---
--- Directory filter rules ---
--- end filters ---
--- start filters ---
--- File filter rules ---
- (^|/)backuptest$
- (^|/)home-assistant\.log$
- (^|/)tts/.*$
--- Directory filter rules ---
- (^|/)tts/.*$
--- end filters ---

If I'm reading that right its start-of-line or leading '/' then the text then end of line $

lol and now i ran a backup and it worked

2020/08/18 20:47:53 DEBUG : home-assistant.log: Excluded

:man_shrugging:

I always checked my set with a get. I wasnt restarting the daemon at all so I'm not sure what I was missing.

Is there a way to get the ExcludeFrom and other options to persist a restart/reboot?

4

You can add the options straight to the rcd.

I'd like to sort out the options passing to the RC - it is unsatisfactory at the moment

5

ha! duh - thanks for the answer, i was being lazy and too focused on the parameters via api.

i'm all set now, thank you.

1 Like
6

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.