What is the problem you are having with rclone?
Inline comments in a "--filter-from" file effectively remove the filter on that line (with directories in my hands). As far as I can tell this behavior isn't documented/warned and doesn't throw errors.
This tool is super powerful, by the way. Thanks! This is obviously a super small thing. I think that this behavior should simply be noted in the documentation.
Run the command 'rclone version' and share the full output of the command.
Discovered on...
rclone --version
- os/arch: linux/amd64
- go version: go1.15.5
...and vererified issue persists on...
rclone.current --version
- os/version: ubuntu 23.10 (64 bit)
- os/kernel: 6.5.0-35-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.23.1
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
SFTP, but I suspect this is storage system agnostic.
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone copy -vv --immutable <origin>:<path> <destpath> --filter-from my_filters.txt --transfers 10 --tpslimit 10 --dry-run --log-file=./mylogs.log
My filter-files
comment_filters.txt
## exclude this dir that would otherwise match
- /dir1/** # inline comment removes this exclusion
# include dir2 and dir3
+ /dir**
# exclude everything else (required)
- *
no_coment_fitlers.txt
## exclude this dir that would otherwise match
- /dir1/**
# include dir2 and dir3
+ /dir**
# exclude everything else (required)
- *
The rclone config contents with secrets removed.
[<remote_name>]
type = sftp
host = <hostename>
user = <usr>
pass = <passwd>
md5sum_command = none
sha1sum_command = none
shell_type = unix
A log from the command with the -vv flag
Inline comment -- dir1 is not excluded, oops
2024/10/18 18:18:51 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "copy" "-vv" "--immutable" "<origin>:<path>" "<destpath>" "-P" "--filter-from" "./test_filters.txt" "--transfers" "10" "--tpslimit" "10" "--dry-run" "--log-file=./inline_comment.log"]
2024/10/18 18:18:51 DEBUG : Using config file from "<path_to>/.config/rclone/rclone.conf"
2024/10/18 18:18:51 INFO : Starting HTTP transaction limiter: max 10 transactions/s with burst 1
2024/10/18 18:18:51 DEBUG : Creating backend with remote "<origin>:<path>"
2024/10/18 18:18:51 DEBUG : sftp://<usr>@<url>:22//<origin+path>: New connection <ip>:<port>-><otherip>:22 to "SSH-2.0-OpenSSH_7.4"
2024/10/18 18:18:51 DEBUG : Creating backend with remote "<destpath>"
2024/10/18 18:18:51 NOTICE: dir3/file3.txt: Skipped copy as --dry-run is set
2024/10/18 18:18:51 DEBUG : sftp://<usr>@<url>:22//<origin+path>: New connection <ip>:<port>-><otherip>:22 to "SSH-2.0-OpenSSH_7.4"
2024/10/18 18:18:51 DEBUG : sftp://<usr>@<url>:22//<origin+path>: New connection <ip>:<port>-><otherip>:22 to "SSH-2.0-OpenSSH_7.4"
2024/10/18 18:18:51 NOTICE: dir1/file1.txt: Skipped copy as --dry-run is set
2024/10/18 18:18:51 NOTICE: dir2/file2.txt: Skipped copy as --dry-run is set
2024/10/18 18:18:51 DEBUG : Local file system at <destpath>: Waiting for checks to finish
2024/10/18 18:18:51 DEBUG : Local file system at <destpath>: Waiting for transfers to finish
2024/10/18 18:18:51 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Transferred: 3 / 3, 100%
Elapsed time: 0.4s
2024/10/18 18:18:51 DEBUG : 29 go routines active
No inline comment -- dir1 excluded properly
2024/10/18 18:17:41 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "copy" "-vv" "--immutable" "<origin>:<path>" "<destpath>" "-P" "--filter-from" "./test_filters.txt" "--transfers" "10" "--tpslimit" "10" "--dry-run" "--log-file=./no_inline_comment.log"]
2024/10/18 18:17:41 DEBUG : Using config file from "<path_to>/.config/rclone/rclone.conf"
2024/10/18 18:17:41 INFO : Starting HTTP transaction limiter: max 10 transactions/s with burst 1
2024/10/18 18:17:41 DEBUG : Creating backend with remote "<origin>:<path>"
2024/10/18 18:17:41 DEBUG : sftp://<usr>@<url>:22//<origin+path>: New connection <ip>:<port>-><otherip>:22 to "SSH-2.0-OpenSSH_7.4"
2024/10/18 18:17:41 DEBUG : Creating backend with remote "<destpath>"
2024/10/18 18:17:41 DEBUG : dir1: Excluded
2024/10/18 18:17:41 NOTICE: dir3/file3.txt: Skipped copy as --dry-run is set
2024/10/18 18:17:41 DEBUG : sftp://<usr>@<url>:22//<origin+path>: New connection <ip>:<port>-><otherip>:22 to "SSH-2.0-OpenSSH_7.4"
2024/10/18 18:17:41 NOTICE: dir2/file2.txt: Skipped copy as --dry-run is set
2024/10/18 18:17:41 DEBUG : Local file system at <destpath>: Waiting for checks to finish
2024/10/18 18:17:41 DEBUG : Local file system at <destpath>: Waiting for transfers to finish
2024/10/18 18:17:41 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Transferred: 2 / 2, 100%
Elapsed time: 0.2s
2024/10/18 18:17:41 DEBUG : 20 go routines active