Incremental (chunked) encrypted files?

Feature
1

I'm interested in being able to do incremental up/download of large-ish (say dozens of megabytes) files where:

  1. The files are encrypted.
  2. "Small" updates to the files on the client do not require re-uploading the whole file.

I believe that just using the crypt remote would involve full re-uploads.

The combination of crypt and chunker seems close, but my understanding is the chunker uses fixed-size chunks, which can lead to terrible misalignment of chunks.

I think what I'm groping for is something like the content based chunking in restic (and other storage systems).

Am I missing something? Is there interest in the community in a feature like this?

2

There is no way for rclone to do this. By design, rclone tries to be as 1:1 between remotes using native objects. Of course there are some remotes that break this but they are the exception.

I floated the idea of a CDC-based wrapper remote before but I do not know golang nor do I think it is a great idea. I also considered writing a Python tool to chunk and upload files for bi-directional sync but again, never had the motivation.

Too many edge cases for it to make sense. I would stick with restic or the like for backups.