Copying into local folders with directory name encryption

What is the problem you are having with rclone?

Copying into an encrypted local remote using directory name encryption has unexpected behaviour with both absolute and relative destination paths.

What is your rclone version (output from rclone version)

rclone v1.51.0
- os/arch: linux/amd64
- go version: go1.13.7

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Fedora 30, 64 bit

Which cloud storage system are you using? (eg Google Drive)

Local encrypted remote

The command you were trying to run (eg rclone copy /tmp remote:tmp)

I started with a new config file:

[local]
type = local

[local-crypt]
type = crypt
remote = local:
password = <generated>
password2 = <generated>

Example 1

Trying to =rclone-copy= into an encrypted absolute path:
\rclone --crypt-directory-name-encryption=true copy /home/user/Downloads/document.pdf local-crypt:/home/user/tmp/rclone-test/encrypted/

Gives this error:

2020/05/16 13:19:55 ERROR : document.pdf: Failed to copy: mkdir /do6rdo23ulutrppoal3nf8kelg: permission denied
2020/05/16 13:19:55 ERROR : Attempt 1/3 failed with 1 errors and: mkdir /do6rdo23ulutrppoal3nf8kelg: permission denied
2020/05/16 13:19:55 ERROR : document.pdf: Failed to copy: mkdir /do6rdo23ulutrppoal3nf8kelg: permission denied
2020/05/16 13:19:55 ERROR : Attempt 2/3 failed with 1 errors and: mkdir /do6rdo23ulutrppoal3nf8kelg: permission denied
2020/05/16 13:19:55 ERROR : document.pdf: Failed to copy: mkdir /do6rdo23ulutrppoal3nf8kelg: permission denied
2020/05/16 13:19:55 ERROR : Attempt 3/3 failed with 1 errors and: mkdir /do6rdo23ulutrppoal3nf8kelg: permission denied
2020/05/16 13:19:55 Failed to copy: mkdir /do6rdo23ulutrppoal3nf8kelg: permission denied

I believe what's happening is rclone encrypts the entire destination path /home/user/tmp/rclone-test/encrypted and attempts to copy document.pdf to it. Therefore /home becomes /do6rdo23ulutrppoal3nf8kelg, which doesn't exist as a root level folder and it fails with permission denied.

The expected behaviour is for only an encrypted version of document.pdf to be placed in the encrypted folder. If I change the directory name encryption to false, rclone correctly places an encrypted file in /home/user/tmp/rclone-test/encrypted.

Example 2

If I perform the same operation but issue the command from /home/user/tmp/rclone-test, the above error does not occur.

cd /home/user/tmp/rclone-test
\rclone --crypt-directory-name-encryption=true copy /home/user/Downloads/document.pdf local-crypt:encrypted/

Again the expected behaviour is for an encrypted copy to be placed in /home/user/tmp/rclone-test/encrypted/, but instead the encrypted file is placed in /home/user/tmp/rclone-test/12oukp3fve3a7hli0tns2mtk14.

It seems the only way I can get the files in the intended destination dir is to cd there first, before issuing rclone copy.

Example 3

The same problem occurs in reverse when I want to decrypt the files, for example through rclone ncdu or rclone mount.

I have to issue the ncdu or mount with exactly the same relative path as I originally used to copy the files into the encrypted store, otherwise rclone doesn't decrypt the files.

Question

I can work around these with multiple encrypted local remotes for multiple hardcoded destination paths, but it would be preferable if I could use a single local-crypt remote that can copy an encrypted file to whatever path I specify on the command line.

I'd appreciate any feedback whether this is intended behaviour, or a bug, or I'm just doing something wrong.

What is it you are trying to do?

local makes your existing directory appear so if you are trying to crypt on top of that, it wouldn't work because you already have something existing there so you can crypt it.

hello,
this does not seem to be a bug.
instead an issue with setting up the local crypted remote.
you seem to be trying to copy encrypted and non-encrypted files to the same folder.
that mixing will create problems.

do you need directory name encryption, if not, then perhaps

RCLONE_CRYPT_FILENAME_ENCRYPTION=false
RCLONE_CRYPT_DIRECTORY_NAME_ENCRYPTION=false
RCLONE_CRYPT_PASSWORD=TJ2tUcsyLRZNguAg2lkJdpRvdGonerAL
RCLONE_CRYPT_PASSWORD2=WWjs0TUU7g2D1lBtnm6Kegt4Pg-gOpNX

./rclone copyto decrypted.txt /mnt/c/path/to/local/folder/encrypted.txt

you seem to be trying to copy encrypted and non-encrypted files to the same folder.
that mixing will create problems.

No, I don't want to have encrypted and non-encrypted files in the same folder. What I'm trying to achieve is just a folder of encrypted files that I can later serve over webdav with rclone.

do you need directory name encryption, if not, then perhaps

Yes, the sticking point is how to achieve a folder of encrypted files with directory name encryption. I have it working without directory name encryption, but the problem arises when I enable that option.

[local]
type = local

[local-crypt]
type = crypt
remote = local:
password =
password2 =

Change that to where you want your folder.

[local]
type = local

[local-crypt]
type = crypt
remote = local:/some/dir
password =
password2 =

Then all your files will be put there encrypted.

Rclone copy dir1 local-crypt:

Change that to where you want your folder.
Then all your files will be put there encrypted.

Yes, I mentioned that in my OP. I can hard code destination directories in my rclone config, but then I'll need a separate remote for every destination dir.

The empty local-crypt directory works fine when I'm not using directory name encryption, I think it should work the same when directory name encryption is enabled.

That's why I suspect it's a bug - the way directory names are encrypted by rclone is interfering with the destination directory. Directory names should only be encrypted below, or within, the destination directory.

When not using directory encryption it can find the files it needs to. But it can't traverse the non-encryptrd names when it's enabled because it can't read them anymore. This is working as expected. You would need to add any encrypted directory anchors as remotes.

Typically what people do is they have a place to store all their encrypted files and they only need one directory for entry. Then they create a encrypted done system within that.

Think about this:

rclone --crypt-directory-name-encryption=true copy /home/user/Downloads/document.pdf local-crypt:/home/user/tmp/rclone-test/encrypted/

Rclone looks at your remote which says use local:/ and encrypt from there.

You tell it copy this PDF to home/user/tmp/rclone-test/encrypted/` which is a folder contained WITHIN the Crypt starting at local:/ it can't create a structure from /. Only root could do that.

If you specify local:/home/user in your comfig it may then work the way you want but you're going to be mixing encrypted and non-encryptrd files because it's going to create a structure from where you root the the encrypted remote.

So if you did the above, and you ran this:

rclone --crypt-directory-name-encryption=true copy /home/user/Downloads/document.pdf local-crypt:/tmp/rclone-test/encrypted/

You home directory will contain the normal stuff AND the encrypted version of tmp/rclone-test

You really shouldn't mix encrypted and non. You can.. and crypt will only list the encrypted files (by design) but I wouldn't recommend it.

If it did that, how would it know how to descend from the root of your remote to the file if you did a rclone ls local-crypt. It would be missing all the directories decending to the target directory.

Thanks for the reply. I am not sure if you don't understand me, or I don't understand you (more likely :slight_smile:)

You really shouldn't mix encrypted and non. You can.. and crypt will only list the encrypted files (by design) but I wouldn't recommend it.

As I said earlier, I'm not trying to mix encrypted and non-encrypted files in the same directory. To explain how I expected things to work, for a command like this:
rclone --crypt-directory-name-encryption=true copy /home/user/Downloads/document.pdf local-crypt:/home/user/tmp/rclone-test/encrypted/

I expected rclone to see I am copying document.pdf, encrypt it and put it in /home/usr/tmp/rclone-test/encrypted.

If I copied a directory like /home/user/Downloads/mydocs, I would expect the directory mydocs and its contents to be encrypted and put in /home/usr/tmp/rclone-test/encrypted.

I don't see how that requires rclone to know anything it doesn't already know, or to traverse directories it can't read. If rclone can encrypt to a target dir, I expect it can decrypt from that target dir too. I also don't expect the full path of the target dir to be encrypted, which is happening.

I don't know the inner workings of rclone encryption so my expectations are quite possibly just misplaced. Regardless, it is looking easier to hardcode a target dir into my config, and then I won't need to specify a target dir on the command line, and then I'll avoid these errors.

as per the docs, that is the way to do it.
https://rclone.org/crypt/#specifying-the-remote

Based on your config in your OP, you are mixing them as you have local: and are trying to encrypt local:

That command is expecting everything after local:crypt to be encrypted and based on your setup, it isn't. It's a non encrypted directory in a crypt remote.

I do understand you. But rclone must encrypt from where your remote is anchored to the directory you specify. The way you are trying to do it, you are mixing encrypted and non since the path between local:/ (root) to /home/user/aaaaa/aaa/aaa must have a encrypted hierarchy.

Thanks for the extra explanation, I understand better now. I will do as recommended and hardcode my encrypted directory in rclone config.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.