Which special characters do I need to escape? why can't rclone handle it?

Help and Support
1

i have a filter file that i generated by another process. using sed it is made to looks like this:

name/**
another name/**
name [with square]/**
[123-123] [456 456]/**

first of all, the line with two sets of square brackets causes rclone to die in the ass:
rclone -v --include="[123-123] [456 456]/**" copy /blah/ remote:blah/

017/08/05 08:03:51 Failed to load filters: bad glob pattern "[123-123] [456 456]/**" (regexp "(^|/)[123-123] [456 456]/.*$"): error parsing regexp: invalid character class range:3-1`

I can escape the square brackets like \[123-123\] \[456 456\]/** and it will not fail, and it will match.

Since this is dynamically generated and rclone can’t handle the special characters, even inside quotes, I need to use sed to escape all those characters. is there a list of all the characters i need to find and escape?

I am using this to make it work for now, but I still get the blob errors on some names:
sed -e 's/.*\///' -e 's/[^a-zA-Z 0-9\-]/\\&/g' -e 's/$/\/**/'

is there a better way?

2

If you know the names of the files in advance then use --files-from to select every file name you need - that might be easier?

The include flags take filter patterns which means that *?[]{} have special meanings. These patters are knnown as file globs.

sed -e 's/.*\///' -e 's/[^a-zA-Z 0-9\-]/\\&/g' -e 's/$/\/**/'

This looks OK to me - what errors are you still seeing?

3

in the end I am using:
sed -e 's/.*\///' -e 's/[][]/\\&/g' -e 's/$/\/**/'

rclone would die with the same bad glob pattern when it would have something like \- or \(asd\) in the exclude file resulting from the inverse alphanumeric sed. however if i only escape the square brackets, everything is fine. i think. for now…

so far i have noticed that {} are no problem, and i guess * are treated as wildcards anyway. just [] messes it up

can --files-from can be used to exclude?

4

No it is just a list of files. I don’t think you can use it with --exclude either.

5

I don’t why I thought curly brackets worked without escaping… this is the sed I am now using and definitely works!

sed -e 's/.*\///' -e 's/[][{*?}]/\\&/' -e 's/$/\/**/'

6

I have a One Drive Personal account and one of the directories uses # character. I cannot specify that directory on command line for listing (lsjson for example). Per comments, in backend/onedrive/replace.go, # is a legal character for One Drive Personal, yet there does not appear to be any code to prevent replace.go replaceReservedChars function from mapping # to FULLWIDTH NUMBER SIGN for One Drive Personal. Is this a bug? Is there a way to specify the # character on rclone command line for One Drive Personal? Using macOS High Sierra which has bash shell.

7

Interesting info on allowed characters with Microsoft technologies:

8

rclone uses the superset of the onedrive for business characters and the onedrive personal to rename them.

If you’ve got an existing directory with a # in then rclone won’t be able to access it. Can you rename it with the unicode equivalent using the onedrive web interface? rclone will be able to access it fine then.