Automatically mount crypted folders "in place" when mounting the parent remote

Hi,

I looked at the docs to see if what I suggest here can already be achieved but didn’t succeed to find something - if I overlooked something I am very sorry :slight_smile:

My Situation: I synchronize stuff from my NAS to ACD, and I want to have some stuff crypted, other stuff not.

Say I have the following structure on my NAS:

<folder root>/public/foo.jpg
                     bar.jpg
                     ...
             /secret/secret.doc
                     private.doc
                     ...

By setting up a config for ACD “acd:” and a crypt config for the folder that contains secret material “secret:” pointing to acd:/secret and syncing accordingly I easily achieve the following on ACD

<folder root>/public/foo.jpg
                     bar.jpg
                     ...
             /secret/<random-gibberish-1>
                     <random-gibberish-2>
                     ...

Which is what I want to achieve on ACD - public parts are not encrypted, private things are.

Now comes my question/feature wish:

For making use of ACD on the road, I mount “acd:” on my laptop via rclone. But to make any use of anything that is below /secret/ I have to mount additionally the “secret:” config to another point on my laptops, because the “acd” mount will of course show everything under /secret/ as random gibberish.

What I would like to see is rclone recognizing, that there are crypt configs for folders somewhere inside of “acd:” and - when mounting acd: - transparently mount the “secret:” at it’s “correct” place inside of the ACD folder tree, essentially giving me exactly the layout as it is on my NAS.

So “secret:” would basically be mounted on top of the acd:/secret folder transparently…

I hope I made myself clear about what I want to achieve :slight_smile:
Regards, and thanks for the very cool work so far!

Greetings, jtl

1 Like

Interesting idea…

You could (and I haven’t tried this so it might not work) mount the crypt inside the first mount, so

mount acd: /mnt
mount secret: /mnt/secret

which would achieve what you want.

If that doesn’t work, you could try unionfs which I think can do similar things…

2016/12/09 23:47:48 mount helper error: fusermount: bad mount point /mnt/secret: Permission denied
2016/12/09 23:47:48 Fatal error: failed to mount FUSE fs: fusermount: exit status 1

This is what happens when trying to mount on secret on top of acd.

I thought about unionfs, but I think it can’t do what I want, as I am not looking for a union of both branches, but the secret folder on ACD should be completely invisible after the secret: mount is done “over” it.

On top, it would be pretty cool not to have to re-build the tree manually, but rclone (and just one rclone instance) recognizing the layout from its config definitions.

Just read the code a bit - you need to use the --allow-non-empty flag on the second mount…

This returned the exact same error message :slight_smile:

This isn’t a rclone issue. This is a permission issue. If you want to layer like that, then you’ll likey need to run it as root via sudo.

sudo rclone mount robacd: 0 --allow-other --gid 1000 --uid 1000 --umask 007 &
[1] 18952
mkdir 0/g
sudo rclone mount robacd-crypt: 0/g --allow-other --gid 1000 --uid 1000 --umask 007 &
ls 0/g | wc -l
7

and if you want to layer ontop of an existing folder (like Nick said) then you additionally need the allow-non-empty flag to allow it.

sudo rclone mount robacd: 0 --allow-other --gid 1000 --uid 1000 --umask 007 &
sudo rclone mount robacd-crypt: 0/cloudm --allow-non-empty --allow-other --gid 1000 --uid 1000 --umask 007 &
ls 0/cloudm | wc -l
7

You are right of course.

But having to do that somehow defeats the original feature request to have a very simple. non-root possibility tp mount the “full” tree. If rclone would internally route some parts of the tree through the crypt layer and some not (according to the config file), there would be no problem with fuse as there would only be one instance of rclone that is running, and only one mount point.

Regards, jtl

Technically, you don’t need root depending on how you want to carve up permissions. But this is a permission problem only. Example:

rclone mount robacd: 0 --allow-other --gid 1000 --uid 1000 --umask 000 &
rclone mount robacd-crypt: 0/cloudm --allow-non-empty --allow-other --gid 1000 --uid 1000 --umask 000 &

I’m not saying that your enhancement is good or bad. Just offering help.

Just a small question regarding the procedures here (sorry if I didn’t see/understand some obvious docs) - for a feature request, is putting it in the forum as a feature the right thing or is putting some issue in github what I am expected to do?

Thanks for any pointers, jtl

Discuss in forum first then make a feature request on github is the preferred procedure.

May I take your “Interesting idea” from your original reply as enough heads up that the idea is worth being saved in a feature request :slight_smile: ?

I think my preferred option would be to make the two mounts work as described in my original answer. That is a more general purpose solution and doesn’t complicate the internals of rclone any more. Can you make that work for you?

If it would work exactly as you describe it (no need for doing it as root, or allow-other or other permission changing stuff) I could live with it being two calls to rclone mount. Having to use --allow-non-empty I don’t see as a problem. But having to use root or playing with permissions is :slight_smile:

The unfortunate side effect would be that in my desktop environment I would still see two (or more - I personally have already two encrypted sub dirs on ACD and there will be more) different mount points that can/must be unmounted separately, which I wanted to avoid with my initial approach.

That is what I’d like to fix, so if you made an issue about that I’d be grateful.

Done here:

Issue on github

1 Like

I have one more question regarding the way you are going to implement the topic.

With your approach, I would (in my case) still have to do 3 rclone mount calls to contruct my tree. No problem with that, I have a script that does that currently for me (just side by side and not mounted “in tree” over each other)

But - to put the mounts into the background, I essentially have something like this (rem1 containing two folders wth crypted content, that is made accessible as two crypted remotes rem2 and rem3)

rclone mount rem1:  /acd/path1 &
rclone mount rem2:  /acd/path2 &
rclone mount rem3:  /acd/path3 &

Note the “&” to put it into the background…

After your enhancement, my script will look something like this (details omitted)

rclone mount rem1: /acd/path1 &
rclone mount rem2: /acd/path1/foo/path2 &
rclone mount rem3: /acd/path1/bar/path3 &

Now I wonder: will this work, or will there be race conditions given by the fact that I put 3 mounts into the background. I already see today that the mounts will not consistently happen in the order I start them (which is not surprising)

Or to put it in other words: If the second mount happens before the first one - will the system just fail or magically somehow the second mount will still shadow the directory of the crypted files in rem1? I somehow doubt that…
As I want to have the mount be in the background, how would I ensure the right order?

My original idea obviously wouldn’t have that problem as one rclone instance would only result in mount for the whole tree…

Hmm, tricky!

I think I’d put a sleep 1 between each mount. Not pretty but it will work!

or better, wait for the mount to really appear, thats what i do when mounting multiple ACD mounts in a bash script. a simple while loop will do the trick. same thing for unmounting

well… the sleep 1 version is… really ugly :slight_smile: and I have been in places with bad connection where I’d rather have needed a sleep 3 to be sure it works…

The while loop… having a tight spinning loop for something that may potentially fail in the sense that the break criteria might never appear… it’s also getting to look like a very complicated and non pretty solution for the desired effect :frowning:

Well of course I could do things like waiting only a certain time, and fail if a mount is not working, unwinding all the already successful mounts prior to the fail. But having to script that in bash doesn’t look like a rewarding job to me.