Wondering if my use case is the correct one for rclone

Help and Support
1

I want to encrypt a few folders on my external drive, then back them up along with some unencrypted data to Backblaze Personal (using Backblaze's own client for backing up).
I don't think the cloud choice really matters here since I'm intending to use Rclone for encryption only as sort of a more stable version of Cryptomator where I would mound the encrypted folders as extra drives, then view and modify the files from there, then back up the encrypted folder full of obfuscated files.

All the guides I've come across treat Rclone as a cloud backup solution which can encrypt along the way, not an encryption tool. Backblaze Personal doesn't allow that (and neither am I interested in using it as such).

Would doing this double the local storage use?

Would using rclone in this manner incur any noticeable performance hit?

2

hello and welcome to the forum,

  • rclone is very flexible, you can use it any way you want.
  • that would not be using double storage as rclone mount does not download all the files to local storage.
  • only if you access a file via the mount, then rclone will downloaded it to rclone's local cache.
    after a period of time, rclone will remove unused files from it local cache.
    so you could rclone mount a 50TB remote, access a 1MB file, and only that 1MB file will be in the rclone local cache,
  • there should not be any noticeable performance hit.
1 Like
3

Sounds pretty neat!

So let's say I access all the files in the mount (e.g. I search for something or use a tagging solution for file navigation and retrieval which would necessitate keeping track of all files and their tags).
Would that double the storage on the system drive or the drive the encrypted folder resides on?

4

it depends on the type of search.

let's use the example of a rclone mount pointing to a remote full of media files, crypted or not.
the total size of all media files in the b2 remote is 50TB.

if you are searching based on file names, file size, modtime, then rclone will not download the contents of files.
there will be a small amount of metadata downloaded and b2 might charge for the api calls and small amount of data transfer.

if you are searching based on the contents of the files, again, it depends on how much data an app requests and thus rclone has to read from b2.

for example, i have a local media server, emby, which needs to scan all the files to create its database.
by default, emby, will scan/read the entire contents of all files, to create thumbnails, previews and whatnot.
therefore, rclone will have to download perhaps the full 50TB of data.

however, i have tweaked emby, when scanning media files, not to create those thumbnails, previews and whatnot.
in this case, emby will only need to scan a small amount of the media file, to get basic metadata info.
in this case, rclone will download just a small amount of data per file.

either way, once emby has scanned the files, it does not need to scan them again.
however, emby, like most media servers, uses scheduled task to re-scan media and that could trigger rclone to download the requested data, which again, can be the entire file or just parts of it.

5

it depends on the type of search.

Like a File Explorer search in Windows.
Since I'm just encrypting a local external drive, the searches will be local.
I'm expecting to see search results in the form of thumbnails, filenames, metadata.
Retaining the ability to search certain file contents (PDF, txt files) would be ideal.

The tagging solution I'm using relies mostly on the filename so it should be similar to a regular filename search.

6

it the crypt is local, then it really does not matter much.

7

Doesn't matter in what way?
Are you saying it won't need extra storage for browsing and searching through the encrypted mount?

8

as i explained, it depends on what type of search and what the search reads.

it also depends on what you plan to do with the files themselves.

if you read only from the files.
as with my emby server, then i do not use the vfs cache, so zero extra storage is required to cache.
when i watch a video, emby asks rclone for the data, rclone reads the data from the internet and feeds it to emby. the data read is not cached at all on local storage.

if you plan to write/modify the files, then you will need to use a vfs cache, which again, will only cache the data your app requires from it.

fwiw, as i do not know your exact use case, for a local crypt, i would not use rclone mount.
i am heavy user of windows bitlocker and in the past, true/veracrypt.

9

Not sure what else I can tell about my usecase.
I want to encrypt a ~5TB folder on an 8TB local drive before backing up that folder plus a couple more folders from that drive as well as from the C drive using Backblaze's client.
The 8TB drive only has around 1TB free space left, and the C drive is a 1TB SSD that's about half full, so I don't really wanna use much (if any) extra space.
The folder is frequently being written to, read from and searched through using File Explorer's search and a filename-based tagging solution.
In other words, it's a pretty "dynamic" folder.
Veracrypt and bitlocker wouldn't work because I would need to reupload an entire container every time I change a small file.

10

in that case, no way would i use use a rclone mount with vfs cache over a crypt remote.

really, that is not correct.

most winboxes i use have bitocker enabled and for sure, any server or craptop i setup, always has the boot drive c:, crypted with bitlocker.
for external storage, like usb drives, use bitlocker to go or veracrypt.

for full system bare metal recovery image backups i use free veeam agent, tho vagent can recover individual files.

for backup of individual files on a bitlocker volume.
fastcopy, 7zip, second copy, of course rclone and any other copy tool.
it it runs on windows, it will backup individual files on a bitlocker and/or veracrypt volume.

11

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.