Is there a way to specify default exclude filters at the remote config level?

Feature
1

Windows, 1.64.2

I'm running into issues with filenames not being supported on the remote when using Microsoft Office on files on an rclone mount, which is causing strange behavior. For example:

2023/11/19 08:44:55 ERROR : Dropbox root '': sync batch commit: failed to commit batch length 1: batch had 1 errors: last error: path/disallowed_name
2023/11/19 08:44:55 ERROR : Documents/~$stDocument.docx: Failed to copy: upload failed: batch upload failed: path/disallowed_name
2023/11/19 08:44:55 ERROR : Documents/~$stDocument.docx: vfs cache: failed to upload try #1, will retry in 10s: vfs cache: failed to transfer file from cache to remote: upload failed: batch upload failed: path/disallowed_name
2023/11/19 08:45:07 ERROR : Dropbox root '': sync batch commit: failed to commit batch length 1: batch had 1 errors: last error: path/disallowed_name
2023/11/19 08:45:07 ERROR : Documents/~$stDocument.docx: Failed to copy: upload failed: batch upload failed: path/disallowed_name
2023/11/19 08:45:07 ERROR : Documents/~$stDocument.docx: vfs cache: failed to upload try #2, will retry in 20s: vfs cache: failed to transfer file from cache to remote: upload failed: batch upload failed: path/disallowed_name
2023/11/19 08:45:28 ERROR : Dropbox root '': sync batch commit: failed to commit batch length 1: batch had 1 errors: last error: path/disallowed_name
2023/11/19 08:45:28 ERROR : Documents/~$stDocument.docx: Failed to copy: upload failed: batch upload failed: path/disallowed_name
2023/11/19 08:45:28 ERROR : Documents/~$stDocument.docx: vfs cache: failed to upload try #3, will retry in 40s: vfs cache: failed to transfer file from cache to remote: upload failed: batch upload failed: path/disallowed_name

At this point Word can't save the file due to a "file permissions error". I need to save the file to a local drive. Then I can't copy the file to the remote even via Windows Explorer, because "the filename is not valid". I have to close the mount, remount it and then I can copy the file.

EDIT: It's worth noting that using --network-mode also screws with Office, because rclone makes it look like a network drive mounted from \\server, and then Office sits there trying to contact "\\server" for a good 30+ seconds before "giving up" and just opening the file.

It looks like I can solve this by adding an exclude pattern to the mount command, like:

rclone mount --exclude "~$*" --vfs-cache-mode full remote: y:

But it's easy to forget to include that exclude statement, and that doesn't solve the issue of potentially needing more of them if some other application plays weird games with files. Eventually I end up needing a fancy script to mount a remote with all of the options required to make it play nice.

It might already exist, but is there a way to specify some sort of "default" exclude patterns for a remote in the configuration, so that any operations done on that remote use the given exclude patterns all the time? (Perhaps an option like --ignore-excludes would bypass it if necessary?) It would make the most sense for this to be remote-specific, not global, since "~$" filenames would be just fine for, say, local syncing or syncing to an SFTP server.

I could actually see a "default options" being useful in other ways too - for example, being able to provide a list of command line options per remote that apply all the time if relevant (and could be overridden somehow if needed). For example, --vfs-cache-mode full being applied all the time for any operation on a given remote where that option has meaning. Or even cosmetic things - --volname "something" always being applied on mount.

It would be "best" if this could be stored in the rclone.conf file since that would allow those settings to carry over if you copy the config file to another system.

2

Not today but --vfs-upload-exclude flag is in the making:

One day it will be ready.

There are default options for every remote flag. E.g. for --vfs-cache-mode it is off. What is or is not relevant unfortunately depends on your requirements. Defaults in general never suit everybody.

3

There are default options for every remote flag. E.g. for --vfs-cache-mode it is off . What is or is not relevant unfortunately depends on your requirements. Defaults in general never suit everybody.

Yeah, I know there's defaults internally, but what I was thinking of was a way to specify your options in the config file. For example:

[myremote]
type = dropbox
token = ...
option_vfs_cache_mode = full
option_exclude_0 = ~$*
option_exclude_1 = .DS_Store
option_progress = true

and so on.

This way your own overrides follow the config and aren't then required on the command line each time you run the command.

It's a quality of life feature so of course it's going to be lower priority, but it would definitely be nice.

4

these are global flags...

so what rclone should do if remote1 has option_progress = true and remote2 has option_progress = false and you do

rclone copy remote1: remote2:

???

remote flags are remote specific and can be put in rclone config file. Global flags are for rclone command

5

fair point... so maybe the scope should be more limited to commands that only use one remote like mount, serve, etc.

The work on vfs excludes should help, but doesn't this just replace --exclude with --vfs-upload-exclude? In other words I still have to remember to put it in my mount command? or am I missing something?

6

Yes you will have to remember it.

To make your life easier if you want to use the same flags with every rclone command simply create rclone alias - not sure how to do this on Windows though as I do not use this OS.

In unix you simply define that e.g.:

myrclone = rclone --exclude "~$*" --flag2 --flag3

and use myrclone command

7

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.