Inclusion of special characters in config name

Initially, I didn't think of requesting this as a feature for Rclone because I was under the impression that the proposed part of the topic was already in place. I was trying to add the email address in the config name, which led me to create an issue under 'Help and support.'

Feature request:
To include special characters (or at least the "@" character) support while creating a config name.

Usability once implemented:
The "@" character can help differentiate between the remotes since it will allow users to include their email addresses (valid for popular cloud services). The current set of characters can allow many combinations, but this particular addition can help in the instance mentioned.

A user can better understand which one drive account is connected to the remote if the name can be odbusiness - user@example.com instead of the current implementation odbusiness.

Reference to the topic mentioned above:
Special characters in config name - Help and Support - rclone forum

The proposed phrasing of the new rule to the existing documentation is mentioned below:

  • May contain the Password Special Characters:

" !"#$%&'()*+,-./:;<=>?@[]^_`{|}~"

  • May contain numbers, letters, _, -, . @ and space. (if the character "@" is only implemented)

I can help make the implementation possible only to a limited scale as I am still learning the tool and in the initial stages of its usage. However, if I can help in any way possible, I am happy to do so.

The current rules are

Valid remote names

Remote names are case sensitive, and must adhere to the following rules:

  • May contain number, letter, _, -, . and space.
  • May not start with - or space.
  • May not end with space.

Starting with rclone version 1.61, any Unicode numbers and letters are allowed, while in older versions it was limited to plain ASCII (0-9, A-Z, a-z). If you use the same rclone configuration from different shells, which may be configured with different character encoding, you must be cautious to use characters that are possible to write in all of them. This is mostly a problem on Windows, where the console traditionally uses a non-Unicode character set - defined by the so-called "code page".

There are quite a few constraints on remote names

  1. easy to type at the command line (must not contain a shell metacharacter for all OSes, can't be !$%*()[]{}<>?| - not an exhaustive list!)
  2. mustn't be a character rclone uses as paths (eg : and / and \)
  3. they can be file names under every OS (can't be /\"*:<>?|)
  4. it must work in the config file (rules out [ and ])

These rules rule out nearly every character in !"#$%&'()*+,-./:;<=>?@[]^_{|}~. The most likely to pass would be @,+^;

If we were to add @ it would need to pass these rules.

  1. OK for unix shells - not sure about Windows - @Ole @albertony ?
  2. OK - rclone doesn't have special meaning for @
  3. I believe that @ can be in file names under Windows, Linux and Mac.
  4. Not sure about the config parser - need to check but likely OK

@albertony - you were involved with the increase from ascii characters - what do you think?

I think @ is safe, also considering Windows: It's a plain ascii symbol, can be in filenames, works in Windows cmd/powershell...

It does have some special meaning in batch files:

In a batch file, the @ symbol at the start of a line is the same as ECHO OFF applied to the current line only.

(Echo - Windows CMD - SS64.com)

But that should not matter, the following two would both work as expected:

echo off
rclone copy dir user@remote
@rclone copy dir user@remote

There may of course be some catch we are forgetting here, but I think it is safe to try out - extend the tests and verify that the parsers handle it.

Agree, it also seems to be safe according to this: Naming Files, Paths, and Namespaces - Win32 apps | Microsoft Learn

The special use in CMD is well spotted, and a reminder that adding special characters may have wider implications that one may initially think. As an example we should probably also consider valid URLs and JSON when used in things like the API or rclone serve combinedRemote:

Excellent analysis!

I would be tempted to add them all to avoid repeating this again in a week when we get a good argument to add e.g. +

That would also make a well rounded list of legal characters, in my eyes:

  • May contain letters, numbers, _, -, +, ., ;, @, ^ and spaces.

Note that use of ; needs quoting in Linux, and PowerShell on Windows, as it is used to separate multiple commands on a single line.

Also ^ needs quoting in Windows Command Prompt, it is actually an escape character:

C:\Temp>echo hello&world
hello
'world' is not recognized as an internal or external command,
operable program or batch file.

C:\Temp>echo hello^&world
hello&world

Argh, I am missing the can of worms emoji.

To me, quoting is kind of given as soon as moving beyond letter, number and _, but using an escape character sounds more scary. Perhaps pull ^ from the list and add "," which I happened to forget - useful for "OneDrive - Frost, Ole"

Yeah, I can agree with that. I didn't say we can not use those symbols, I haven't really thought it through if this would lead to real problems, just wanted to make sure we are aware of the possibility at least.

Perhaps pull ^ from the list and add ","

Sure, then I don't have to "think it through"... :slight_smile: Perhaps better to be on the safe side, and also I think ^ is not exactly the symbol that is missed the most...

I had these in mind but was unsure whether the implementation could be made possible. Point 4 was something that didn't cross my mind.

Essentially, the @ character itself can tremendously improve the config naming convention. Although the inclusion of the possible characters during the testing period can rule out the possible problem causing characters.

Something that I have in my mind too. Whenever I can help, I will be around :blush:

If what I understood is correct, the new list would be something like given below:

  • May contain letters, numbers, _, -, +, ., @ and spaces.

And ,

So; adding the three symbols @, , and + to the existing _ , - , .

Then a question if we should also include ^ (probably not) and ; (not sure)

Thinking more about it, probably not a good idea. It is used separator in connection strings.

Example: gdrive,use_trash:some/folder

Agree, wait a couple of days and we probably have come up with examples to reject both.

This is an example misreading is easily possible: remote;some/folder

I will not miss ^ or ;

Well spotted! I should have put , in my original list. If we allowed , it would make parsing config strings impossible.

I want to keep the potential metacharacters in config names to the minimum so that we don't interfere with unknown systems!

We added . recently which allowed config names to be domain names, adding @ will allow them to represent nearly all email addresses so that seems like a good change.

The utility of adding + is probably marginal, though you do get email addresses with + in.

Yes, definitely a sleep on it decision!

:+1:

@yolobnb Sounds like there is consensus that adding @ and + is a great idea :slightly_smiling_face:

I suggest you create an rclone feature request on GitHub: https://github.com/rclone/rclone/issues/new?template=Feature.md

1 Like

@Ole @ncw @albertony I have created an issue on Github as per the suggestion. Hopefully, we can see the feature in the next update :blush:

1 Like

Thanks Yolo!

Here is a link to easily track status:

It is good to be optimistic and it looks like @albertony is very kind with this small change. It will however typically take a lot longer without contributing with sponsorship or development. There are 700+ open issues and you can get an impression of the number of issues implemented per release by looking in the change log.

2 Likes

Thanks, and well said. And that link contains a link to beta builds, so if @yolobnb would like to contribute with testing that would be appreciated. As discussed we are not 100% confident there isn't some caveat so the more regression testing the better.

I can certainly help for testing purposes. Thanks for standing by till the feature was updated.

Thanks, @albertony, for merging the necessary commits to the repo and doing it at lightspeed. I will upgrade to the beta release now and start testing.

I am a bit confused about the beta version. Is the command below the right one to be used?

rclone selfupdate --beta --version v1.62.0-beta.6693.038099fde.remote-name-ext

Hm, I'm not really sure about the selfupdate argument, but seems to me it doesn't consider beta builds from branches, i.e. only considers beta builds from master at root of https://beta.rclone.org/. But its as simple as manually downloading the zip with your browser, extracting to a temp folder and start testing from there. It will then re-use your existing config file if it is in a standard location. If you would have it completely stand-alone you can also customize it to use its own config file, see https://rclone.org/docs/#config-config-file.

1 Like