Waiting until an rclone mount is ready

Help and Support
1

Hi! I'm trying to use rclone mount as part of a larger system, where I'd first like to mount a remote, then start using it immediately from some other code.

The problem I'm facing is that after starting rclone in the background, the mount point takes some unknown amount of time to initialize. In the meantime I need to wait before running any code which depends on the mount being available.

The problem is knowing how long to wait; ideally we'd wait for an event from rclone and there would be an OS-independent way to detect when the mount point is ready. So far I've found that the systemd integration seems to be approximately the right thing; if I supply a NOTIFY_SOCKET and read from it with netcat:

nc -lk -U -u /some/socket/path

followed by running rclone separately:

export NOTIFY_SOCKET=/some/socket/path
rclone mount remote_name: mountpoint --read-only --vfs-cache-mode=full

after doing this we'll see READY=1 come down the socket.

I'm not sure this will work on windows though, as the use of "unixgram" in the following doesn't look promising. However, it's necessary to use "unixgram" to get a UDP socket opened for systemd compatibility, as in the C system call socket(AF_UNIX, SOCK_DGRAM, 0).

It appears that connectionless UDP unix domain sockets as in the nc command above are a somewhat unusual thing, possibly not accessible via libuv which I'm using using for IO.

So what to do? In principle it seems possible to generalize the systemd stuff in a small way such that it would work on windows. Alternatively I was hoping that rclone rc might help out in some way, but I wasn't sure exactly how to do that. Perhaps I'll try rclone rc mount/mount next and hope that it blocks until the mount is ready.

Environment

In this case I've been testing remote_name with a google drive remote, but I doubt the particular remote config makes much difference.

rclone version information:

rclone v1.53.1
- os/arch: linux/amd64
- go version: go1.15
2

No it won't work on windows as the support isn't compiled in on Windows.

The rc call mount/mount will return at pretty much the same time that the notification would have been sent so I think that will work :slight_smile:

You could use something old school like listing the mount and waiting for a file to show up. On Windows the mount point doesn't exist until after the mount so waiting until the mountpoint shows up on Windows will work.

1 Like
3

Thanks, out of the available options, this does seem attractively simple. For now, I've found a hack which achieves something similar on linux - simply call stat() on the mount point and its parent directory and compare the device numbers. If they're equal, the mount is not yet initialized. Other than needing to poll it seems like a reasonable solution.

(Further research indicates that comparing device numbers seems to be a standard trick. For example python's os.path.ismount does it this way.)

4

That is a good trick :slight_smile:

5

I have

rclone mount "$src" "$dst" $flags
# wait for the mount to be ready
until findmnt --mountpoint "$dst" --mtab >/dev/null; do :; done

at the end of my rclone mount wrapper script. It's easy to understand and works very well. findmnt is part of the util-linux package so if you're on Linux, you probably have it installed already.

3 Likes
6

I didn't know about findmnt - nice one :slight_smile:

7

Yes that's a great option and I'd use it if I was in a linux shell scripting environment. My problem was that I'm trying to use rclone as a data backend for a cross-platform Julia package. So I can't easily rely on external tools like findmnt and I'd rather avoid starting external processes in any case. So the stat() trick fits my particular constraints quite well.

(By the way, I made a build of Rclone available via the Julia package manager https://github.com/JuliaBinaryWrappers/Rclone_jll.jl)

8

Nice! I don't quite know what that means but it looks cool :slight_smile:

Do you want to put a link to it on the rclone wiki? Maybe on https://github.com/rclone/rclone/wiki/Third-Party-Installers ?

9

Haha :slight_smile: The Julia language comes with a package manager which knows how to distribute Julia packages, but also a cross platform cross compilation system to build and distribute a wide variety of precompiled binaries. (Not difficult for Go of course but a trickier prospect for C++ and Fortran which many important numerical libraries relevant to Julia users tend to be written in.)

Anyway, I made a build recipe, so installing rclone in a Julia REPL on any platform is as simple as Pkg.add("RClone_jll").

Done!

10

That is very clever. I thought Julia was just a language, I didn't realise it has an almost OS level package manager as well.

:slight_smile:

Great.

I'd like to tweet about this... If you want to compose a tweet then you can @ me and I'll retweet, or if you aren't a Twitter user I'll make something up :slight_smile:

11

Sorry I missed this. I don't have a twitter account, but go for it!

it has an almost OS level package manager as well.

Yes, thinking about it like apt/macports/chocolaty etc etc etc is a fairly good model. It comes with a strong semver-based dependency resolver, so it's quite reliable for producing working project configurations even when packages are evolving. Pkg.jl is one of the good parts :slight_smile:

1 Like
12

Tweet sent: https://twitter.com/njcw/status/1335894422203084800 :slight_smile:

I have Julia on the list of language to look at when I have time.

Thank you!

1 Like
13

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.