Is there a way to use a relative path to point to the GSuite service account .json file?
I've tried path/file.json and ./path/file.json, but only absolute path works (i.e /this/is/an/absolute/path/file.json)
It's not a huge issue, but it would simplify the sharing of my rclone config between different OSes.
I'd remove it from the config and just make an environment variable.
https://rclone.org/docs/#environment-variables
I have not specifically tried it with a service account, but I would surmise it would work like any other item.
Thanks, but I'm not totally sure how that would make it easier?
Afaik I still would need to enter a custom path on different systems, and it would be another step, other than making rclone look for relative paths (from where the config file is locate).
You have a profile on each system and you have a custom environment variable in each system that points to the right location for the service file.
I use a custom config location in my .bashrc
# RClone Config file
RCLONE_CONFIG=/opt/rclone/rclone.conf
export RCLONE_CONFIG
But that still does not make it easier, unless I'm not getting something here.
In my case I have a centralized rclone config on an NFS share that different VMs use at the same time. I mainly use Linux, and as long as I set the same mount point on the different VMs it works fine.
The "problem" starts if I ie. use macOS and get a different mount point (/Volumes/ unless you manually mount the server at a non default mountpoint).
If the path would be relative it would work regardless of mount point. Docker does this. You can write an absolute path or use ./path/to/somewhere that uses the docker-compose.yml location as the relative path starting point.
Each user has a profile on each system.
For each user, make the environment variable the path you need to point to the service file.
So user1 on server a may point to /home/user1/bob.service and user2 on server b may point to /home/userb/bob.service
It's not a i.e systemd service I'm talking about, it's a google cloud project service account that is shared accross different users/VMs.
My point is that I would like to avoid specifying anything unique on different systems, all I want is to be able to run rclone with a shared config on an NFS share.
i.e on Linux I would issue:
rclone --config '/mnt/server/path/rclone.conf' <cmd>
and on macOS:
rclone --config '/Volumes/path/rclone.conf' <cmd>
But that would fail on macOS since the service account file reference in the rclone config needs an absolute path like service_account_file = /mnt/server/path/projectname-12jhk12812jk.json and I would have to change that to service_account_file = /Volumes/path/projectname-12jhk12812jk.json on macos.
If there was relative paths based on rclone.conf location, I could simply use service_account_file = ./path/projectname-12jhk12812jk.json and it would work out of the box on different systems, regardless of mount points.
Yeah on Linux I use:
felix@gemini:~$ env | grep CONFIG
RCLONE_CONFIG=/opt/rclone/rclone.conf
and on my Mac, I use.
textere@seraphim ~ % env | grep CONFIG
RCLONE_CONFIG=/Users/textere/Documents/rclone.conf
You can do the same thing with environment variables for the service account file location.
I didn't miss your point as I said you can use this as a work around. If you want to explore using a relative path in rclone, we can have @ncw opine on it and you can submit a feature request if it is possible or not.
Maybe it's me who was missing your point then. 
I guess your workaround solves the "one config file to rule them all" problem, and just overriding whatever you need using a enviroment variable.
I tested with export RCLONE_CONFIG_<REMOTE>_SERVICE_ACCOUNT_FILE=/Volumes/path/to/serviceaccounts/file.json and that worked.
That is slightly better than creating multiple config files, but I would really like it to work with relative paths. Hopefully that's not too hard to fix.
I've learned it's never your fault as I explained it bad so I can always improve! If I flip the blame to you, I never get better.
Sweet!
We can let Nick answer that bit as some things I think are easy are hard but things I think are hard are easy so I never know.
I'm not sure why it doesn't work with relative paths... I would have thought it would work straight out of the box...
I tried this and it worked fine as a relative path either path/file.json or ./path/file.json worked for me.
Weird, it does not work here. It does work if I execute the rclone command in the config directory.
ie. this works:
Path: /mnt/server/rclone-conf-dir/
rclone --config /mnt/server/dir/rclone.conf lsd remote:
-1 2020-05-28 11:09:04 -1 88j1258dbkk6ltsqi1kt76ik14
-1 2020-05-27 09:59:12 -1 ag2j4insftm8urj2973jke1ku8
-1 2020-05-18 23:24:51 -1 nh3n2l2it6k3pfta4di3lnfjl4
-1 2020-05-18 23:25:16 -1 pa9nk8oc9h5saam6vtenbghi64
-1 2020-05-18 23:24:42 -1 pnr90tm29o33ad9ukotpv344t0
This does not work:
Path: ~ (or any other non rclone conf path)
rclone --config /mnt/server/dir/rclone.conf lsd remote:
2020/05/28 14:49:11 Failed to create file system for "remote:": drive: failed when making oauth client: error opening service account credentials file: open ./serviceaccounts/rclone-23221-671a1238748bc7.json: no such file or directory
Tested on both macos and linux (ubuntu server), same result.
Ah I think I see the confusion. The path will be relative to the current directory you execute rclone in, not relative to the config file or config directory.
I suspect what you'd really like is do be able to write ~/.rclone/serviceaccounts/rclone-23221-671a1238748bc7.json to be relative to the users home directory?
That would be straight forward to implement.
Nope, not in this case, but I can see the use for that.
If it's possible to make it relative to the config location, it would solve both cases. Docker does this, and that makes it possible to deploy a docker directory with references inside the directories.
How does docker specify that in the config?
I could set an env var from within rclone then you could use ${_RCLONE_CONFIG_DIR} in the config file - that would also be straight forward.
Yes, I guess that would work.
In dockercompose config you just use ./relative/dir
I guess that would be easy enough too!
Some of the backends use env.ShellExpand to expand the config file variables
$ go doc env.ShellExpand
package env // import "github.com/rclone/rclone/lib/env"
func ShellExpand(s string) string
ShellExpand replaces a leading "~" with the home directory" and expands all
environment variables afterwards.
It would be dead easy to add this.
@nle do you want to work on something like this?
Glad it's dead easy, but I have no experience developing in anything other than html, css and a little bit of bash scripting.
OK I had a go at this!
Can you have a go with
https://beta.rclone.org/branch/v1.52.0-020-g33a9be7f-fix-config-files-beta/ (uploaded in 15-30 mins)
This will enable you to use ${RCLONE_CONFIG_DIR} in your config.
Thanks for this, but unfortunately it looks like it does not work.
Seems like it uses ~/.config/rclone as the default even if I specified the config file with --config (or with an enviroment variable)
2020/06/02 17:01:37 Failed to create file system for "<remotename>:": failed to make remote <remotename>:"" to wrap: drive: failed when making oauth client: error opening service account credentials file: open /home/<user>/.config/rclone/serviceaccounts/rclone-281891-0f9db71dd10bc.json: no such file or directory
[1] + 22222 exit 1 rclone mount --config /mnt/<server>/backup/_rclone/gsuite/rclone.conf
In the config it's specified like this:
[<remotename>]
type = drive
client_id = <redacted>
client_secret = <redacted>
scope = drive
service_account_file = ${RCLONE_CONFIG_DIR}/serviceaccounts/rclone-281891-0f9db71dd10bc.json
team_drive = <redacted>