Is matching encryption for different remotes possible?

What is the problem you are having with rclone?

I'm using rclone to encrypt and upload to OpenDrive. The upload speed isn't great so I thought of running the encryption to a local destination then just use the OpenDrive app to upload. I created an alias for the local destination, then a crypt remote pointing to said alias with the same password1 and password2 (copy/paste from where I saved them) I used for the crypt remote for OpenDrive remote. Do a few files to test, the hash of the encrypted file in OpenDrive is different from the encrypted one at the local destination. I also tried matching up the obscured passwords in the config file, that didn't work either. How would I be able to make sure they're encrypted exactly the same so regardless of how I upload or download them, I can still encrypt/decrypt them all the same?

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.0

  • os/version: Microsoft Windows 10 Pro for Workstations 21H1 (64 bit)
  • os/kernel: 10.0.19043.1826 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.18.3
  • go/linking: static
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

OpenDrive and local

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy

The rclone config contents with secrets removed.

[OpenDriveG]
type = opendrive
username = 
password = 
chunk_size = 100Mi

[COpenDriveG]
type = crypt
remote = OpenDriveG:FLIR
filename_encryption = off
directory_name_encryption = false
password = 
password2 = 

[CLocalTemp]
type = crypt
remote = LocalTemp:Temp
filename_encryption = off
directory_name_encryption = false
password = 
password2 = 

[LocalTemp]
type = alias
remote = 

Encryption with any modem CPU only adds a very small percentage to the process and most likely, you won't even notice it.

For sharing the same crypt details, you need password/salt combination. You can copy the one rclone.conf crypt or enter in the same details for all the prompts and that would work as well.

I'm not referring to the CPU usage, it's the bandwidth/connection that I'm worried about. I have plenty of head room in my connection, it's OpenDrive that's intermittenly drop connections and stay dropped for probably 15 minutes at a time. Jobs that gave me 3 hours ETA end up taking over 12 hours easily.

Yes I have the password/salt combo, I saved them. When I run the config again for the local destination crypt, I copy/paste the password/salt combo, but the obscured strings in the config file doesn't match up with the one I have for OpenDrive crypt.
I try copy/paste the same obscured strings from OpenDrive crypt to local crypt, that didn't get me matching hash for encrypted files either.

Should I just turn off password obscure in the config file to make sure they're both match and try again, I feel like that's the 'when all fails try unencrypted telnet' solution and I'd rather not go there? Or is there a better way?

You'd notice bandwidth even less than you would CPU.

That would be the same crypt or non crypt as that sounds like OpenDrive just isn't that good. I don't use it so I can't really share much other than other posts I've read, it doesn't get great reviews.

I'm not quite following. I copy my rclone.conf across many machines with configs and never had any issues with crypt's not working. You can type in the same passwords/salt in as well.

Can you share the error when you try to use the crypt?

That would be the same crypt or non crypt as that sounds like OpenDrive just isn't that good. I don't use it so I can't really share much other than other posts I've read, it doesn't get great reviews.

Yea I figured it's OneDrive's end, hence thinking if I use their client, it'll have slightly better result? At least I wouldn't look nearly suspicious? I just want to have consistent encryption regardless of what upload/download method I choose to use. I use them for their pricing as one of my 2 cloud backup, but that's neither here nor there.

Can you share the error when you try to use the crypt?

There's no error. The crypt runs fine, but when I compare the file in OpenDrive uploaded via rclone's remote crypt vs the one encrypted to a local destination, the hashes don't match.

I copy my rclone.conf across many machines with configs and never had any issues with crypt's not working. You can type in the same passwords/salt in as well.

I'm wondering if it may have something to do with the destination, that's the only difference in here. Maybe rclone uses my OpenDrive password together with the specified password/salt somehow that the local destination doesn't? I'm just wondering if others have tried to do something similar, using rclone more for encrypt/decrypt and rsync on windows, and have better success than I have.

The hashes are not supposed to match.

The hash would change each time you upload the same file as that's how crypt works.

felix@gemini:/data/rsnapshot$ rclone copy /etc/hosts gcrypt: -vvv
2022/08/09 12:52:45 DEBUG : Setting --config "/opt/rclone/rclone.conf" from environment variable RCLONE_CONFIG="/opt/rclone/rclone.conf"
2022/08/09 12:52:45 DEBUG : rclone: Version "v1.59.1" starting with parameters ["rclone" "copy" "/etc/hosts" "gcrypt:" "-vvv"]
2022/08/09 12:52:45 DEBUG : Creating backend with remote "/etc/hosts"
2022/08/09 12:52:45 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2022/08/09 12:52:45 DEBUG : fs cache: adding new entry for parent of "/etc/hosts", "/etc"
2022/08/09 12:52:45 DEBUG : Creating backend with remote "gcrypt:"
2022/08/09 12:52:45 DEBUG : Creating backend with remote "GD:crypt"
2022/08/09 12:52:46 DEBUG : hosts: Need to transfer - File not found at Destination
2022/08/09 12:52:47 DEBUG : hosts: md5 = 20f5f68c2a775a0e3a153bf7a7c8100e OK
2022/08/09 12:52:47 INFO  : hosts: Copied (new)
2022/08/09 12:52:47 INFO  :
Transferred:   	        375 B / 375 B, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         1.3s

2022/08/09 12:52:47 DEBUG : 6 go routines active
felix@gemini:/data/rsnapshot$ rclone delete gcrypt:hosts
felix@gemini:/data/rsnapshot$ rclone copy /etc/hosts gcrypt: -vvv
2022/08/09 12:52:53 DEBUG : Setting --config "/opt/rclone/rclone.conf" from environment variable RCLONE_CONFIG="/opt/rclone/rclone.conf"
2022/08/09 12:52:53 DEBUG : rclone: Version "v1.59.1" starting with parameters ["rclone" "copy" "/etc/hosts" "gcrypt:" "-vvv"]
2022/08/09 12:52:53 DEBUG : Creating backend with remote "/etc/hosts"
2022/08/09 12:52:53 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2022/08/09 12:52:53 DEBUG : fs cache: adding new entry for parent of "/etc/hosts", "/etc"
2022/08/09 12:52:53 DEBUG : Creating backend with remote "gcrypt:"
2022/08/09 12:52:54 DEBUG : Creating backend with remote "GD:crypt"
2022/08/09 12:52:54 DEBUG : hosts: Need to transfer - File not found at Destination
2022/08/09 12:52:55 DEBUG : hosts: md5 = be790418e87c6f2034ba7f9fc091ba0a OK
2022/08/09 12:52:55 INFO  : hosts: Copied (new)
2022/08/09 12:52:55 INFO  :
Transferred:   	        375 B / 375 B, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         1.1s

2022/08/09 12:52:55 DEBUG : 6 go routines active

I thought you were saying you couldn't view the files.

rclone cryptcheck

wait... whaa...t?? Pardon my stupidity here but I thought symetric encryptions are supposed to produce the same thing given the same key? If so then why would the hashes be different every time?

If that's the case then the obscured strings in the config file, they're not supposed to match either even if the password/salt pair provided are the same?

I think there are some posts on that in the forums as that's not my area of expertise other than I know they won't match.

No, if you always got the same key / result, encryption would be very broken :slight_smile:

felix@gemini:~$ echo "secretpassword" | rclone obscure -
MrAOGCcNnoSNbod-rjioWWa-9Gxrq2RQWn65Ijjw
felix@gemini:~$ echo "secretpassword" | rclone obscure -
5cz1I2oPV2OF62w0DF_d_kB_cbxM9-gNDFVEHJ-C
felix@gemini:~$ echo "secretpassword" | rclone obscure -
FXGq-A4ynrDT8VgqR4f7cx3oQWhpWyWasdR_ZFQS
felix@gemini:~$ echo "secretpassword" | rclone obscure -
era2BfGNDiNTkZALAVLRtF8w2LmiwwKtRSEBHPQO

Yeap, I found this post in which your replies totally blow my mind again. I didn't even know MD5 is long broken. I've been using checksum/hash interchangably and apparently that's incorrect. It is correct that hashing from the same content will produce the same hash, but crypt/bcrypt doesn't produce the same encrypted text every time, even with the same password/salt pair and I think I should leave it at that...

Thank you for clarifying. I guess people didn't ask this because they know already...

1 Like

I learn something almost every day around here so if you aren't sure, ask is my motto.

I rather ask a "stupid" question and learn something that not ask at all!

(I'm also wrong here or there too so that happens too:) )

yea I think you're being modest when you say this isn't your area of expertise...
I learnt a new word, 'nonce' through this.
I'll test the encryption again in a different method.

And I'm with you on the ask stupid question rather than fixing stupid mistake, I work in IT, I presume you too? The dangerous users are the ones that think they know everything but actually don't.

Thank you for your help!

1 Like

IT and a lot of years of Taekwondo as well so lots and lots of questions..

A nonce ("number once") is a number that should only ever be used once. Or at least once within a time window/comms channel. The idea behind it is to detect a "replay attack"... basically "hey, I've already seen this nonce; something bad is happening; reject!"

Hashing has a similar concept; it's normally called a "salt". A good hash system will generate a new salt each time it's called, so if you want to hash "hello" and do it twice then you get two different results. So that hashes can be used for passwords (e.g. a login database) the salt is stored with the hashed value. The idea is to make it harder to create "rainbow tables" (a list of all possible hashes) because the word "hello" could be hashed 2^64 different ways.

Encryption also has a similar concept. Here it's commonly called "initialization vector" (IV). This creates an initial permutation of state of the encryption algorithm. So if you encrypt the word "hello" it will be different each time. This prevents simple comparisons ("this encrypted file is the same as that encrypted file so it must have the same content"). The IV is also stored with the encrypted data so the decryption process knows where to start.

We can see that nonce, salt, IV are all very related, they're ideally unique. Sometimes you'll hear IVs and salts called nonces because they share similar attributes (random values, hopefully not repeated).

What does all this mean? You encrypt the same file twice, you'll get different results. You can't compare the encrypted files, you can only compare the unencrypted files.

I thought the word was derived from a nonce word which is

a word invented for a particular occasion or situation

Though given the fact that very few people have ever heard of a "nonce word" your definition is much easier to understand!

The word nonce is also British slang for a pedophile which is unfortunate, but there you go!

"Nonce word" comes from the (archaic? I don't think I've come across this outside of older books) phrase "for the nonce" (meaning, "for the time being" or similar). Nonce words are typically used multiple times during a conversation as a stand-in for some concept, to avoid having to describe the concept every time. A useful nonce word may even get adopted into the language :slight_smile:

In contract, a cryptographic nonce focuses on the "once" part. We only use this number once (eg once in a communication channel to prevent replay attacks; eg once as an IV to prevent two blocks being encrypted the same way). It's very possible that "number once" is a back-formation, but that's pretty much how it's described these days.

And, yeah, I'm an Englishman (living in America); "nonce" was a playground chant back when I was a young kid (far too many years ago). I get some amusement explaining the word to Americans :slight_smile:

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.