Running rclone inside a network namespace

Made1908 · July 31, 2023, 10:09am

What is the problem you are having with rclone?

I am trying to mount a webdav folder inside a network namespace to route all traffic through a VPN. The mount seems to work with no error but the mounted folder is always empty. When I am mounting outside of the namespace it works perfect. The problem is only in the mounting because I can see all my webdav files when running the command "rclone-linux ls" inside the namespace.

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.0-DEV

os/version: ubuntu 22.04 (64 bit)
os/kernel: 6.1.0-1017-oem (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.18.1
go/linking: static
go/tags: none

Which cloud storage system are you using? (eg Google Drive)

WebDav

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

(1) Inside Namespace

sudo ip netns exec ****** sudo -u **** ./rclone-linux -vv mount WD: /home/****/test2 --dir-cache-time 10s --allow-other

(2) Outside Namespace

./rclone-linux -vv mount WD: /home/****/test2 --dir-cache-time 10s --allow-other

The rclone config contents with secrets removed.

[WD]
type = webdav
url = *************
vendor = other
user = *******
pass = **********************

A log from the command with the `-vv` flag

(1) (2)

2023/07/31 12:04:08 DEBUG : rclone: Version "v1.59.0-DEV" starting with parameters ["./rclone-linux" "-vv" "mount" "WD:" "/home/*****/test2" "--dir-cache-time" "10s" "--allow-other"]
2023/07/31 12:04:08 DEBUG : Creating backend with remote "WD:"
2023/07/31 12:04:08 DEBUG : Using config file from "/home/*****/.config/rclone/rclone.conf"
2023/07/31 12:04:08 DEBUG : found headers:
2023/07/31 12:04:08 INFO  : webdav root '': poll-interval is not supported by this remote
2023/07/31 12:04:08 NOTICE: webdav root '': --vfs-cache-mode writes or full is recommended for this remote as it can't stream
2023/07/31 12:04:08 DEBUG : webdav root '': Mounting on "/home/*****/test2"
2023/07/31 12:04:08 DEBUG : : Root:
2023/07/31 12:04:08 DEBUG : : >Root: node=/, err=<nil>

kapitainsky · July 31, 2023, 10:25am

update your rclone to the latest version please. There is no point in troubleshooting outdated version

Made1908 · July 31, 2023, 10:36am

Done

rclone v1.63.1
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 6.1.0-1017-oem (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.20.6
- go/linking: static
- go/tags: none

Probelm is still there

kapitainsky · July 31, 2023, 12:07pm

I do not use namespaces so no idea what can be wrong.

But maybe you could change approach and instead of namespaces just use iptables:

run rclone as dedicated user e.g. rclone
set iptables to route all traffic from rclone user over your VPN network interface e.g. tun0

Made1908 · July 31, 2023, 1:08pm

This will unfortunately not work becuse a network namespace is isolated and you can't see the interface from the outside. But thanks for the idea.

kapitainsky · July 31, 2023, 1:09pm

I meant not to use namespace at all. I would be surprised if your issue is rclone related (if rclone ls works) - this can be something to do with fuse and namespaces etc.

Made1908 · July 31, 2023, 1:20pm

Unfortunately I have to use a namespace for my task. And yes after debugging a little bit more I think the problem is somewhere on fuse beeing unable to mount inside a network namespace even if this theoretically should be possible without a problem.

kapitainsky · July 31, 2023, 1:23pm

what about to run e.g. rclone serve webdav inside namespace.

And mount it outside of namespace - this way all traffic to remote would go over namespace but mount/fuse would live outside.

Made1908 · July 31, 2023, 2:20pm

I am trying to understand how this works. Do I understand it correctly?

(1) Create the server inside namespace

sudo ip netns exec webserver sudo -u **** rclone serve webdav WD:

from that I get

2023/07/31 16:12:42 NOTICE: webdav root '': --vfs-cache-mode writes or full is recommended for this remote as it can't stream
2023/07/31 16:12:42 NOTICE: webdav root '': WebDav Server started on [http://127.0.0.1:8080/]

(2) Mount this server outside of namespace

rclone -vv mount WD: /home/*****/test3 --dir-cache-time 10s --allow-other --bind 127.0.0.1

When I am now trying to acess the folder the following happens:

2023/07/31 16:16:15 DEBUG : rclone: Version "v1.63.1" starting with parameters ["rclone" "-vv" "mount" "WD:" "/home/*****/test3" "--dir-cache-time" "10s" "--allow-other" "--bind" "127.0.0.1"]
2023/07/31 16:16:15 DEBUG : Creating backend with remote "WD:"
2023/07/31 16:16:15 DEBUG : Using config file from "/home/*****/.config/rclone/rclone.conf"
2023/07/31 16:16:15 DEBUG : found headers:
2023/07/31 16:16:15 INFO  : webdav root '': poll-interval is not supported by this remote
2023/07/31 16:16:15 NOTICE: webdav root '': --vfs-cache-mode writes or full is recommended for this remote as it can't stream
2023/07/31 16:16:15 DEBUG : webdav root '': Mounting on "/home/*****/test3"
2023/07/31 16:16:15 DEBUG : : Root:
2023/07/31 16:16:15 DEBUG : : >Root: node=/, err=<nil>
2023/07/31 16:17:24 DEBUG : /: Attr:
2023/07/31 16:17:24 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2023/07/31 16:17:24 DEBUG : /: ReadDirAll:
2023/07/31 16:17:24 DEBUG : : Dir.ReadDirAll error: couldn't list files: Propfind "************": dial tcp 127.0.0.1:0->**.**.**.**:443: connect: invalid argument
2023/07/31 16:17:24 ERROR : IO error: couldn't list files: Propfind "**************": dial tcp 127.0.0.1:0->**.***.*.**:443: connect: invalid argument

What did I do wrong?

kapitainsky · July 31, 2023, 2:26pm

(1) is ok IMO

now outside of the namespace

add to rclone.config:

[netns.webdav]
type = webdav
url = http://127.0.0.1:8080
vendor = other

you might have to replace 127.0.0.1 with whatever is your outside facing IP of namespace

and then

rclone mount netns.webdav: mountpoint

kapitainsky · July 31, 2023, 2:49pm

Here is an example how to do this - different problem as it is about how to run torrent client inside namespace

but conceptually it is the same as in your case - how to run rclone serve inside namespace but access it from outside

Made1908 · July 31, 2023, 3:30pm

Perfect that worked

kapitainsky · July 31, 2023, 3:44pm

would you mind to share your solution? If you could create post in "Howto Guides" category it would be very useful for others trying to achieve the same.

system · August 30, 2023, 3:45pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.