I need to make an rclone sync to a remote, and use the --compare-dest or --backup-dir flags. I want the sync to be encrypted with file name encryption, so I need to use a crypt backend. However, the path used by the crypt backend needs to be different for --compare-dest (or --backup-dir) to that the sync needs to go to. In effect, the prefix of each path is unencrypted.
My backend remote might be a sftp one. My encrypted remotes need to NOT encrypt backups/full and backups/differential.
I've tried defining different remotes, either crypt or alias, and this isn't allowed (Failed to sync: parameter to --copy-dest has to be on the same remote as destination) - the check is overly broad (func SameConfigArr() just checks names match).
I'm out of ideas.
What is your rclone version (output from rclone version)
That would be challenging - I've substantially changed my script since I posted. My specific failure occurred when defining two remotes, say, --compare-dest differential: /some/source/path full: where remotes were generated on the fly and appended (by shell script) to a local conf file*, something like
Attempt 1
; The data below is appended at runtime by my script
[differential]
type = alias
remote = encrypted:backups/differential
[full]
type = alias
remote = encrypted:backups/full
Attempt 2
[unecrypted]
type = local; or anything else
; The data below is appended at runtime by my script
[differential]
type = crypt
remote = unecrypted:backups/differential
...
[full]
type = crypt
remote = unecrypted:backups/full
...
The fundamental limit is the check in func SameConfigArr(), which wants names to match, rather than doing a deeper structural comparison.
Aside: I have now integrated my crypt backends using Keychain Access on OS X, but it's painful. If my script ever works, I'll share it.
*Aside: multiple conf files would make life so much easier, as would using a line based format friendly to POSIX rather than INI and being able to define multiple remotes on the command line. rclone's a superb tool, with an outstanding reputation, which is why I wanted to try it, but it's not a script friendly as rsync. I'm on the fence with it; not supporting hard links and symbolic links is a major need that's missing for proper file system backups and replacing rsync for local copies. It's winning feature is encryption (although it doesn't seem to have any way to ease the rotation of encryption keys).
Yes. The deeper structural comparison is hard though. It would have to be implemented by each backend probably as they are the only ones that know enough about their config file.
Did you check out the connection string way of defining a remote? That ticks all the boxes above I think.
Rclone does support symlinks. It does also support hardlinks as it will copy the files just fine, it just duplicates the storage! I don't know of many cloud storage backends which have the concepts of hardlinks. Google drive used to, but they took that away recently.
You have to re-encrypt the data if you want to change the key. Rclone doesn't have the idea of user keys which unlock a master key so that you can rotate the user keys.
It maybe should... However if a user key has been compromised then the master key may have been compromised too.