RClone mount into docker produces error. bindfs does not

zoickx · July 3, 2021, 4:53pm

SETUP INFO

What is your rclone version (output from `rclone version`)

rclone v1.55.1
- os/type: linux
- os/arch: amd64
- go/version: go1.16.3
- go/linking: static
- go/tags: none

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Debian 10, 64 bit, OpenStack VPS

Linux <hostname> 4.19.0-17-cloud-amd64 #1 SMP Debian 4.19.194-2 (2021-06-21) x86_64 GNU/Linux

Which cloud storage system are you using? (eg Google Drive)

Local. (originally: rclone crypt over backblaze b2)

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone mount /mount-from-host /var/lib/docker/volumes/test/_data --uid 33 --gid 33

The rclone config contents with secrets removed.

No config file.

A log from the command with the `-vv` flag

This is the log after the 5 commands I described in the rclone subsection.

user@host $ rclone mount /mount-from-host /var/lib/docker/volumes/test/_data --uid 33 --gid 33 -vv
2021/07/03 16:39:59 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2021/07/03 16:39:59 DEBUG : rclone: Version "v1.55.1" starting with parameters ["rclone" "mount" "/mount-from-host" "/var/lib/docker/volumes/test/_data" "--uid" "33" "--gid" "33" "-vv"]
2021/07/03 16:39:59 DEBUG : Creating backend with remote "/mount-from-host"
2021/07/03 16:39:59 INFO  : Local file system at /mount-from-host: poll-interval is not supported by this remote
2021/07/03 16:39:59 DEBUG : Local file system at /mount-from-host: Mounting on "/var/lib/docker/volumes/test/_data"
2021/07/03 16:39:59 DEBUG : : Root:
2021/07/03 16:39:59 DEBUG : : >Root: node=/, err=<nil>
2021/07/03 16:40:01 DEBUG : /: Attr:
2021/07/03 16:40:01 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2021/07/03 16:40:01 DEBUG : /: ReadDirAll:
2021/07/03 16:40:01 DEBUG : /: >ReadDirAll: item=3, err=<nil>
2021/07/03 16:40:01 DEBUG : /: Attr:
2021/07/03 16:40:01 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2021/07/03 16:40:01 DEBUG : /: Lookup: name="file-from-host.txt"
2021/07/03 16:40:01 DEBUG : /: >Lookup: node=file-from-host.txt, err=<nil>
2021/07/03 16:40:01 DEBUG : file-from-host.txt: Attr:
2021/07/03 16:40:01 DEBUG : file-from-host.txt: >Attr: a=valid=1s ino=0 size=5 mode=-rw-r--r--, err=<nil>
2021/07/03 16:40:09 DEBUG : /: Attr:
2021/07/03 16:40:09 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2021/07/03 16:40:11 DEBUG : /: Attr:
2021/07/03 16:40:11 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2021/07/03 16:40:12 DEBUG : /: ReadDirAll:
2021/07/03 16:40:12 DEBUG : /: >ReadDirAll: item=3, err=<nil>
2021/07/03 16:40:12 DEBUG : /: Attr:
2021/07/03 16:40:12 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2021/07/03 16:40:12 DEBUG : /: Lookup: name="file-from-host.txt"
2021/07/03 16:40:12 DEBUG : /: >Lookup: node=file-from-host.txt, err=<nil>
2021/07/03 16:40:12 DEBUG : file-from-host.txt: Attr:
2021/07/03 16:40:12 DEBUG : file-from-host.txt: >Attr: a=valid=1s ino=0 size=5 mode=-rw-r--r--, err=<nil>
2021/07/03 16:40:17 DEBUG : /: Attr:
2021/07/03 16:40:17 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2021/07/03 16:40:18 DEBUG : /: Attr:
2021/07/03 16:40:18 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2021/07/03 16:40:26 DEBUG : /: Attr:
2021/07/03 16:40:26 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2021/07/03 16:40:27 DEBUG : /: Attr:
2021/07/03 16:40:27 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>```

My problem X is described here. TLDR: I would like to mount an rclone remote from the host into a docker container. I am successful in so far as it is accessible from the container's root user, but as soon as any finer-grained access control is involved, things break unexpectedly.

My problem Y, with which I come to this forum is the fact that rclone mount appears to behave differently to other FUSE mounts (such as bindfs) in my case. I will present a clean local example.

Suppose I have an rclone remote, say a local directory /mount-from-host
I want to mount it into docker, in such a way that it is owned by user www-data inside the container
I will use docker volumes: bind mounts seem to make no difference, regardless of propagation
Running everything as root on the host, for simplicity
Running the debian container, to provide a minimal example

rclone

Mount the remote into the volume

rclone mount /mount-from-host /var/lib/docker/volumes/test/_data --uid 33 --gid 33

Check the mount on the host

user@host $ ls -al /var/lib/docker/volumes/test/_data
total 5
drwxr-xr-x 1 www-data www-data    0 Jul  3 15:52 .
drwx-----x 3 root     root     4096 Jul  3 14:41 ..
-rw-r--r-- 1 www-data www-data    5 Jul  3 15:40 file-from-host.txt

Run docker with volumes; check the mount from the container, as root

user@host $ docker run -it -v test:/mount-in-container \
                       --user root debian \
                       ls -al /mount-in-container
total 5
drwxr-xr-x 1 www-data www-data    0 Jul  3 15:52 .
drwxr-xr-x 1 root     root     4096 Jul  3 15:57 ..
-rw-r--r-- 1 www-data www-data    5 Jul  3 15:40 file-from-host.txt

Please note the correct uid, gid and permissions are set for the directory.

Run docker with volumes; check the mount from the container, as user www-data

user@host $ docker run -it -v test:/mount-in-container \
                       --user www-data \
                       ls -al /mount-in-container
ls: cannot access '/mount-in-container': Permission denied

Furthermore, the directory looks has a bunch of question marks in ls - a sign of improperly set execute bit:

   user@host $ docker run -it -v test:/mount-in-container \
                       --user www-data debian \
                       ls -al /mount-in-container
ls: cannot access '/mount-in-container': Permission denied
total 72
drwxr-xr-x   1 root root 4096 Jul  3 16:06 .
drwxr-xr-x   1 root root 4096 Jul  3 16:06 ..
-rwxr-xr-x   1 root root    0 Jul  3 16:06 .dockerenv
drwxr-xr-x   2 root root 4096 Jun 21 00:00 bin
(...)
d?????????   ? ?    ?       ?            ? mount-in-container
(...)

bindfs

For comparison, try the exact same steps, but mounting FUSE with bindfs insead of rclone mount.

Mount the remote into the volume

bindfs /mount-from-host /var/lib/docker/volumes/test/_data/ -u www-data -g www-data

Check the mount on the host

user@host $ ls -al /var/lib/docker/volumes/test/_data
total 12
drwxr-xr-x 2 www-data www-data 4096 Jul  3 15:55 .
drwx-----x 3 root     root     4096 Jul  3 14:41 ..
-rw-r--r-- 1 www-data www-data    5 Jul  3 15:40 file-from-host.txt

Run docker with volumes; check the mount from the container, as root

user@host $ docker run -it -v test:/mount-in-container \
                       --user root debian \
                       ls -al /mount-in-container
total 12
drwxr-xr-x 2 www-data www-data 4096 Jul  3 15:55 .
drwxr-xr-x 1 root     root     4096 Jul  3 16:20 ..
-rw-r--r-- 1 www-data www-data    5 Jul  3 15:40 file-from-host.txt

Run docker with volumes; check the mount from the container, as user www-data

user@host $ docker run -it -v test:/mount-in-container \
                       --user www-data debian \
                       ls -al /mount-in-container
total 12
drwxr-xr-x 2 www-data www-data 4096 Jul  3 15:55 .
drwxr-xr-x 1 root     root     4096 Jul  3 16:20 ..
-rw-r--r-- 1 www-data www-data    5 Jul  3 15:40 file-from-host.txt

Conclusion

As you see, the mount created with bindfs behaves differently (and more reasonably if you ask me) to rclone, which generates an error.

Am I just missing some flag in the rclone mount command? C:
I tried playing around with permissions, to no effect.

zoickx · July 3, 2021, 4:59pm

I have a hunch this might belong in the "Suspected Bug" category.
However, seeing as this setup is somewhat convoluted, I don't feel I have enough experience with docker and stuff to mark it as such.

Half-expecting to be pointed to a one-step-guide for correctly mounting rclone remotes into docker. Couldn't find one. Plenty of discussion about mounting from the rclone docker container into the host, but not the other way around.

Animosity022 · July 3, 2021, 5:03pm

When you use fuse based mounts, you need --allow-other so you can have other users access the user space mount.

zoickx · July 3, 2021, 5:12pm

Ha-ha, simple as that. Thank you kindly @Animosity022.

Adding --allow-other to the rclone mount command fixes the issue.

That said though..who is the "other user" in this case? Clearly it doesn't have to do with filesystem permissions?

Animosity022 · July 3, 2021, 5:16pm

Not file systems permissions per se.

Generally, if you have a fuse based file system, that's in the user space and only the user can see it.

So something like:

felix@gemini:~$ rclone mount gcrypt: /home/felix/test --daemon
felix@gemini:~$ cd test
felix@gemini:~/test$ ls
jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv    mounted    
felix@gemini:~/test$ sudo su -
root@gemini:~# cd /home/felix
root@gemini:/home/felix# cd test
-bash: cd: test: Permission denied
root@gemini:/home/felix# ls
ls: cannot access 'test': Permission denied

To have it accessible for another user, you have to use allow-other. Once it's accessible to another user, that's where permissions come into play.

zoickx · July 3, 2021, 5:23pm

I see! A helpful example. Filesystem in user space. All makes sense now. Thanks x2!

system · July 6, 2021, 5:23pm

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.