Precedence rules for config?

i077 · July 6, 2020, 8:10pm

I have a general question about precedence rules for rclone configuration. I'm extending a NixOS module for restic to add rclone functionality, and in it I'm providing options to set both environment variables and read from a configuration file.

My question is this: Suppose I have a remote called remoteA, and in rclone.conf I set it to be of type local. Now suppose I set an environment variable in my shell with RCLONE_CONFIG_REMOTEA_TYPE=s3. (This isn't the most realistic example, but it illustrates my point.) When I run rclone to interact with remoteA, rclone thinks it's an S3 remote, meaning the environment variable takes precedence over the value set in the config file. Is this intended behavior?

Are values set in environment variables supposed to take precedence over/override the same values set in a config file?

What is your rclone version (output from `rclone version`)

1.52.1

Which OS you are using and how many bits (eg Windows 7, 64 bit)

NixOS, x86-64.

ncw · July 6, 2020, 9:39pm

Yes they are

If you want to see the precedence exactly then take a look at

github.com

rclone/rclone/blob/b2f4f52b642ed6b9d70849156e93145d16956a9f/fs/fs.go#L1279


      
          
          // ConfigMap creates a configmap.Map from the *RegInfo and the
          // configName passed in.
          //
          // If fsInfo is nil then the returned configmap.Map should only be
          // used for reading non backend specific parameters, such as "type".
          func ConfigMap(fsInfo *RegInfo, configName string) (config *configmap.Map) {
          	// Create the config
          	config = configmap.New()
          
          	// Read the config, more specific to least specific
          
          	// flag values
          	if fsInfo != nil {
          		config.AddGetter(&regInfoValues{fsInfo, false})
          	}
          
          	// remote specific environment vars
          	config.AddGetter(configEnvVars(configName))
          
          	// backend specific environment vars

From more specific to less specific

flag values
remote specific environment vars
backend specific environment vars
config file
default values

I note that rclone may be frustrating for NixOS because it writes tokens back to the config file.

There has been some discussion on GitHub about writing to a separate file and leaving the config file as read only.

i077 · July 6, 2020, 9:59pm

Thanks! I was looking through the codebase for a bit to see if I could find it myself, but only managed to find where options were declared, not where they were merged. Appreciate the link to the code.

The docs at Documentation didn't quite make it clear how precedence was handled, so this helps clear things up. Maybe include that in the page?

Ah, so if a user did supply a config file, they should supply one that root or the rclone user can write to? If so, I'll make a note of that in the module.

ncw · July 6, 2020, 10:22pm

Good idea!

Yes. Rclone doesn't write back to the config file for all backends, but for ones that use tokens it does.

Here is the issue I was thinking of: rclone 1.49.1 requires mutable configuration file · Issue #3643 · rclone/rclone · GitHub

system · September 5, 2020, 6:22pm

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.

Precedence rules for config?

What is your rclone version (output from rclone version)

Which OS you are using and how many bits (eg Windows 7, 64 bit)

What is your rclone version (output from `rclone version`)