How does rclone/rclone implement distributed, redundant, and encrypted storage of the same file object in different back-end remote storage in the case of multiple storage back-end configurations

ai gives the following description, but it is still very rough. I don’t know if I can give more detailed and clear examples

The replication function of Rclone can be used to realize the distributed, redundant and encrypted storage of the same file object in different back-end remote storage.

The following is an example of a possible configuration:

  1. Create multiple storage backends: configure multiple storage backends according to Rclone's official documentation, and assign aliases to them respectively. For example, create two storage backends:

[s3-1]
type = s3
access_key_id = ...
secret_access_key = ...
region = ...

[s3-2]
type = s3
access_key_id = ...
secret_access_key = ...
region = ...

  1. Create an encryption backend: Create an encryption backend and bind it to multiple storage backends. For example, the following command will create an encryption backend named my-crypt and bind it to the two previously created S3 storage backends:

rclone config --config= create remote-name crypt
s3-1:/MyData
s3-2:/MyData
my-crypt
--crypt-password="password"
--crypt-remote=""
--crypt-filename-encryption
--crypt-directory-name-encryption
--crypt-obfuscate

In this command, s3-1 and s3-2 are the names of the previously created storage backends, is the location of the Rclone configuration file, and password is the encrypted password. Note that the --crypt-filename-encryption and --crypt-directory-name-encryption flags enable encryption of file and directory names, and the --crypt-obfuscate flag generates random file and directory name prefixes for enhanced data protection.

  1. Copy data: After creating an encryption backend, you can use Rclone's copy command to copy the data stored locally to the encryption backend, and store the encrypted data on multiple storage backends respectively. For example:

rclone copy /path/to/local/data my-crypt:/MyData

This command will replicate the encrypted data to multiple storage backends. Because after the data is encrypted, even if it is stored in different storage backends, the information of the original data will not be disclosed. Distributed storage and redundant storage can be achieved by storing the same data in multiple backends.

Sounds like you want a union remote.

You can already use a union remote to copy to multiple backends at the same time based on whatever policy you want.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.