Docker: Start rclone automatically with multiple mounts and WebGUI

piscator · November 4, 2020, 11:25am

I am just starting out with rclone using docker on a QNAP and have successfully mounted a pCloud drive using the following tutorial: https://rclone.org/install/#install-with-docker

I am now trying to add multiple mounts and the webgui with a single command, but I am not sure how to do that.

I have now the following.

docker run \
    --name=rclone \
    -v /share/docker/rclone/config:/config/rclone \
    -v /share/docker/rclone/data:/data:shared \
    -v /etc/passwd:/etc/passwd:ro \
	-v /etc/group:/etc/group:ro \
	-p 5572:5572 \
    --device /dev/fuse \
	--cap-add SYS_ADMIN \
	--security-opt apparmor:unconfined \
    rclone/rclone \
	mount --vfs-cache-mode full pcloud:/ /data/pcloud &

Version:

rclone v1.53.2
- os/arch: linux/amd64
- go version: go1.15.3

Suggestions how I could add additional mount commands and/or the WebGUI (rcd)

piscator · November 4, 2020, 11:52pm

I have found a way to do it with "rclone rc"
It anyone is interested, here is how:

docker run \
    --name=rclone \
    -v /share/docker/rclone/config:/config/rclone \
    -v /share/docker/rclone/data:/data:shared \
    -v /etc/passwd:/etc/passwd:ro \
	-v /etc/group:/etc/group:ro \
	-p 5572:5572 \
    --device /dev/fuse \
	--cap-add SYS_ADMIN \
	--security-opt apparmor:unconfined \
    --restart unless-stopped \
    -d rclone/rclone:latest \
    rcd --rc-web-gui --rc-addr :5572 --rc-web-gui-no-open-browser --rc-user rclone --rc-pass rclone

sleep 5

docker exec rclone rclone rc mount/mount fs=pcloud: mountPoint=/data/pcloud vfsOpt='{"CacheMode": 3}' --rc-user=rclone --rc-pass=rclone
docker exec rclone rclone rc mount/mount fs=dropbox: mountPoint=/data/dropbox vfsOpt='{"CacheMode": 3}' --rc-user=rclone --rc-pass=rclone
docker exec rclone rclone rc mount/mount fs=googledrive: mountPoint=/data/googledrive vfsOpt='{"CacheMode": 3}' --rc-user=rclone --rc-pass=rclone

Note that vfsOpt='{"CacheMode": 3}' stands for '--vfs-cache-mode full'

ncw · November 5, 2020, 10:37am

I'll make a note to make it so you can use the string there too! the 3 is not very friendly is it

piscator · November 5, 2020, 1:46pm

Thanks. That took me quite a bit of time to figure that one out. Probably, it is better to use the --json interface. That is a bit more comprehensive.

I am running into another issue. When I want to run rclone in docker as another user (non admin) as explained in: https://rclone.org/install/#install-with-docker, there is a problem with the cache directory as the /etc/passwd is inherited from the host (home does not exist) I thought of changing the cache directory with --cache-dir, but with the "rclone rc" command, it seems not to have any effect (unless I am doing something wrong )

I start the container as follows. The --cache-dir option works here (without it I get an error)

docker run \
    --name=rclone
    --user $PUID:$PGID \
    -v $VOLUME_CONFIG:/config/rclone \
    -v $VOLUME_DATA:/data:shared \
    -v /etc/passwd:/etc/passwd:ro \
    -v /etc/group:/etc/group:ro \
    -p 5572:5572 \
    --device /dev/fuse \
    --cap-add SYS_ADMIN \
    --security-opt apparmor:unconfined \
    --restart unless-stopped \
    -d rclone/rclone:latest \
    rcd --rc-web-gui --rc-addr :5572 --rc-web-gui-no-open-browser --cache-dir /data/.cache --rc-user $USERNAME --rc-pass $PASSWORD

However when I mount with one of the following commands

docker exec rclone rclone rc mount/mount fs=pcloud: mountPoint=/data/pcloud vfsOpt='{"CacheMode": 3}' --cache-dir /data/.cache --rc-user $USERNAME --rc-pass $PASSWORD

docker exec rclone rclone rc mount/mount --rc-user=$USERNAME --rc-pass=$PASSWORD --json '{ "fs":"pcloud:", "mountPoint":"/data/pcloud", "vfs-cache-mode":"full", "cache-dir":"/data/.cache" }'

The option "cache_dir" seems to have no effect (it uses the default):

2020/11/05 13:29:13 ERROR : error reading available plugins: Error creating : /share/homes/username/.cache/rclone/webgui/plugins/config

piscator · November 5, 2020, 5:01pm

I was confusing the permissions on the docker container with the wanted permissions on the target file system. Still, if you are running the docker container with the "--user $PUID:$PGID" option on any other user besides the root user, you get very strange behavior:

not all the remotes in your rclone.conf are visible (only the first one?)
you cannot mount the container with "rclone rc mount/mount"

Anyway, after digging deeper in the forum posts, I came to the following solution to do the following

Mount cloud storage for a specific user / group.
Modified the default permissions (user: all, group: read, world: none) (umask 027 --> integer 23)
Mount multiple cloud storages.
Everything via an installation of rclone via docker (which is preferable for me, since it is running on a NAS).

The solution:

AUTH="--rc-user=$USERNAME --rc-pass=$PASSWORD"

docker run \
    --name=rclone \
    -v $VOLUME_CONFIG:/config/rclone \
    -v $VOLUME_DATA:/data:shared \
    -v /etc/passwd:/etc/passwd:ro \
    -v /etc/group:/etc/group:ro \
    -p 5572:5572 \
    --device /dev/fuse \
    --cap-add SYS_ADMIN \
    --security-opt apparmor:unconfined \
    --restart unless-stopped \
    -d rclone/rclone:latest \
    rcd --rc-web-gui --rc-addr :5572 --rc-web-gui-no-open-browser --cache-dir /data/.cache $AUTH

sleep 5

mkdir -p $VOLUME_DATA/pcloud $VOLUME_DATA/dropbox $VOLUME_DATA/googledrive
docker exec rclone rclone rc options/set $AUTH --json '{"vfs": {"CacheMode": 3, "UID": '$PUID', "GID": '$PGID', "Umask": 23}, "mount": {"AllowOther": true}}'
docker exec rclone rclone rc mount/mount fs=pcloud: mountPoint=/data/pcloud $AUTH
docker exec rclone rclone rc mount/mount fs=dropbox: mountPoint=/data/dropbox $AUTH
docker exec rclone rclone rc mount/mount fs=googledrive: mountPoint=/data/googledrive $AUTH

ncw · November 6, 2020, 4:17pm

piscator:

However when I mount with one of the following commands

docker exec rclone rclone rc mount/mount fs=pcloud: mountPoint=/data/pcloud vfsOpt='{"CacheMode": 3}' --cache-dir /data/.cache --rc-user $USERNAME --rc-pass $PASSWORD

docker exec rclone rclone rc mount/mount --rc-user=$USERNAME --rc-pass=$PASSWORD --json '{ "fs":"pcloud:", "mountPoint":"/data/pcloud", "vfs-cache-mode":"full", "cache-dir":"/data/.cache" }'

The option "cache_dir" seems to have no effect (it uses the default):

You'll need to add the --cache-dir onto the main rclone command running the rc server I think, or set the global option with the rc.

Looks like you've got it sorted now

piscator · November 6, 2020, 6:28pm

Thanks for tip. --cache-dir was not working here because I executed the docker container with a '--user' parameter that was not root. Lots of weird things happened, so I just leave it to run under root.

Can you actually set it through rclone rc options/set? I could not find a parameter.

rclone rc options/get --rc-user=$USERNAME --rc-pass=$PASSWORD | grep Cache
                "WritebackCache": false
                "CacheMaxAge": 3600000000000,
                "CacheMaxSize": -1,
                "CacheMode": 3,
                "CachePollInterval": 60000000000,
                "DirCacheTime": 300000000000,

I also was wondering about the units of time for e.g. CacheMaxAge. It looks like to be nanoseconds? (default is 1h = 3600s, and it is listed as 3600 with 9 additional zeroes).

ncw · November 7, 2020, 2:37pm

I just checked the code, yes you are right these can't be set by options/set. This is an oversight!

Yes it is nanoseconds. I plan to allow users to pass a string like the command line args take, eg "5s" but that doesn't work yet.

system · January 7, 2021, 10:37am

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.