It is a follow up on #637 which last activity is in 2020.
From what I see the current crypt remote isn't secure aginst metadata leaks. File size can be calculated by sutracting the header and signature, and the directory stucture is untouched except encrypting the name. Combining the both can make educated guesses of what kind of data are being stored.
While it is still usable, I wonder if rclone team is still interested in improving the "crypt" remote given the issue mentioned is quite stale IMO.
To make the crypt work like that it will be necessary to store metadata files with the mapping of files to names.
It is possible of course but increases complexity and slows things down. You'd probably want to have a local dB which you updated from the remote and tried to keep in sync.
The crypt design is a careful compromise of speed Vs security.
We've almost merged a cryptomator backend which chooses different tradeoffs. You could try that.
Thanks for the reply. I had a quick look at Cryptomator's Threat Model which still suffers matadata leaks. Aside from compatibility with mobile clients and without local storage requirement, I don't see additional security benefits of implementing Cryptomator as a remote while "crypt" have the same problems.
Maybe I'm bias, but the design of restic from my untrained eyes are quite rock solid as I'm using it hourly to backup my computer. It doesn't really slow things down IMO but for sure it increases complexity.
I use restic to back up my computer (via rclone of course!). However I like to have a backup (which I do less often) where I store 1 file to 1 object on the remote system to reduce fallout in the case of a problem, and I back up all my photos etc 1:1 using rclone.
restic is very clever, but it is complicated. There is something very reassuring about that 1:1 model that rclone uses.
What metadata would you like to see plugged?
Here are some which leak some info at the moment with crypt
file size - can be recovered
directory structure - some info about directory structure but no names
file names - some info about file name lengths but no names
We could obscure 1 by adding some random data on the end of the file. 2 would need an index file which increases the risk when stuff goes wrong, but we'd implement an fsck function to rebuild the index(es). 3 would be fixed with the index file too.
So yes we could do this at the cost of a larger corruption fallout.
You might say I'm greedy, but I would like to get rid all of them.
I just viewed gocrytfs's threat model and breifly read it's audit. gocryptfs have a similar threat model to Cryptodome and 'crypt' remote where some metadata are not protected. Additionally, the audit revealed that it can't protect against an active attacker (the Dragon/Dropbox scenario). Since all three shares a similar threat model, I would believe the parts of audit's concludion would applys to Cryptodome and 'crypt'. As remotes in rclone are mostly provided by a cloud service, aka someone else computer, it is hard to not include them as a potential threat.
It is undeniable that rclone current 1:1 model is very reassuring, and I understand restic is complicated, but I think rclone can take inspirations and references from it to model a new or improve 'crypt' remote. They solve the index problem beautifully and I doesn't have any corruptions yet. Depulication and snapshots can be omitted. Delete and overwritting files would be a problem if occured concurrently but can use locks as if restic does to solve.
Yes there will be a larger corruption fallout, but that's all about trading risks.
It is fantastic idea. I can easily see people interested in hardened crypt2 implementation. But it is also serious project to do it right - starting with proper design. When you have some prototype working I would be the first to try and test. Overall it sounds ambitious but definitely doable for determined person or team. When working on this please include files' hashes. It is one small but important features missing in current crypt.
To make it trusted it would require also proper security audit - so probably some sponsoring is required to cover cost of external audit.
Thanks for supporting the idea. I would like to take on this task but unfortunately I'm not confident or experienced enough to propose a design nor author an implementation, especially in serious crypto that will be used by many. I even can't get my head around the design of restic, let alone designing one.
Cryptomator's dev lefted a comment on #1647 proposing a common open source cloud-optimized encryption standard. Although there isn't any details yet, if the standard can match the security level to defend aginst an active attacker, I think it worth the wait since only the implementation need to be worried but the design can be proven secure (via audit) and increase interoperability. @ncw should know about this but there wasn't any update since the end of February this year after him asking Cryptomator dev for a new invite.
Hopefully then you can find people ready for this task. At least now it is documented and some passer by might find it interesting to try. We never know.
I did think about reusing restic code and make a backend, but I quickly gives up the idea becuase if that's the case, why not just use restic directly when it also support rclone?
Furthermore, does rclone needs snapshots and deduplication functionality? I think that's beyond what a storage backend should do and make things too complex.
I found a new encrytion file system named CryFS, which will meet the security level as I mentioned, and is designed for cloud use from the start. It also have a master thesis to proof its security but it is way to diffcult for me to understand. I think it is a better candidate. However, it is written in C++ so more work might be needed to implement.