Server drives toasted. Cannot figure out how to restore previously encrypted files to get going again. Trying to use the "copy" command to retrieve the files from a specific Backblaze bucket (compumatter-net-svr) to our new server.
If we try the rclone copy one way, it ends up download everything but they are encrypted "bin" files. If we change which rclone conf section is being referenced, it throws the error shown below. We are in an emergency state and feel confident we have the correct passwords and salt
Run the command 'rclone version' and share the full output of the command.
'''
rclone v1.60.0
os/version: ubuntu 22.04 (64 bit)
os/kernel: 5.15.0-50-generic (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.19.2
go/linking: static
go/tags: none
'''
Which cloud storage system are you using? (eg Google Drive)
Backblaze
The command you were trying to run (eg rclone copy /tmp remote:tmp)
Not sure, but perhaps it is just your config settings for directory and filename encryption that are wrong.
Try this command:
rclone lsf data-drive:compumatter-net-svr
If the directory names are readable then we can deduce that the crypted data was created with directory_name_encryption = false. I guess this is the situation based on your log.
If the filenames are readable and ends with .bin then we can deduce that it was created with filename_encryption = off. I guess this is the situation based on your log.
You can easily verify my guesses with this command before updating your crypt config:
Thank you Ole. Indeed those are both true observations.
Strangely, when I first ran the command
''' rclone lsf data-drive-crypt: --crypt-directory-name-encryption=false --crypt-filename-encryption=off '''
It indeed returned some unencrypted image / jpg files and gave me great hope.
I tried to apply those flags --crypt-directory-name-encryption=false --crypt-filename-encryption=off to my copy command but got errors in the log file like
Now strangely enough just running your original command
'''rclone lsf data-drive-crypt: --crypt-directory-name-encryption=false --crypt-filename-encryption=off'''
Results in a similar error
''' Failed to create file system for "data-drive-crypt:": failed to make remote "data-drive:compumatter-biz-svr" to wrap: failed to authorize account: failed to authenticate: Unknown 401 (401 unauthorized)'''
So I went to backblaze and regenerated keys, ran b2 authorize-account 34e3d... 00124f1a265...
which went properly and without errors.
Running the command rclone lsf data-drive:compumatter-net-svr now still returns the error Failed to create file system for "data-drive:compumatter-net-svr": failed to authorize account: failed to authenticate: Unknown 401 (401 unauthorized) where it did not initially...
Good move to test the underlying remote directly, unfortunately I don't know B2 and can't help with this. My only suggestion would be to try creating a new remote from scratch e.g. data-drive2:
OK, my guesses about the settings for directory and filename encryption were correct, so now you can list the files correctly. This is a bit like seeing the name outside on an archive box.
Next we want to look inside the box and decrypt the content to make it usable, and that is where you see these errors:
They tell you that the decryption ends in gibberish (that is the decrypted data failed to give meaning/authenticate).
This can have three reasons:
The file is corrupt
The content wasn't encrypted (it was made using no_data_encryption = true)
The encryption password(s) are wrong
Ad 1) When this happens to all of your files then it is unlikely to be file corruption.
Ad 2) This can be tested by trying a command something like this:
Perhaps you copied you passwords directly into the config file without obscuring them? See how to obscure them here: rclone obscure
If this also doesn't help, then my best advise is to create a new crypt from scratch e.g. data-drive-crypt2 and then enter the below information to make it usable without any extra flags:
Ole, I will try these commands. Thank you. This screen shot from backblaze has .bin on the end of all the files. It seems to confirm that they are in fact encrypted.
In addition, I opened a simple downloaded txt file and this was inside of it.
'''RCLONE áÕߧìGþ»öJ=”Ëïrýkeª¬yÚhæa6»á¬s³Ímœçy´XÄp¢4òêø:½ïÖ¢mk`wëÖ³v'''
So that seems certain
I have tried what you suggested. The first command results in the following return:
'''
1.JPG
12-33-70.pdf
2.JPG
3.JPG
4.JPG
5.JPG
6.JPG
7.JPG
LocalSettings.php
bridgechristianfellowship.com-le-ssl.conf
bridgechristianfellowship.com.conf
certbot-results.txt
cm_word_4ya.2019.01.07.sql
dam-heading-logo.svg
dam-heading-logo.zip
data-drive/
heading.logo.svg
intro-image.jpg
resources/
servermatter.rsp
servermatter.zip
working/
'''
The second command
rclone copy --config=rclone-test.conf data-drive-crypt2: test-recovery2
Resulted in: in a lot of error messages. I've created a short video that puts you in the drivers seat
The ERRORs you see are all due to the configured passwords not matching the passwords used when encrypting the files. The NOTICEs on removal are just rclone cleaning up the files containing gibberish due to wrong passwords.
Not much I can do from the drivers seat with a wrong set of keys (in a car with fully secured ignition).
The last chance is that you are using obscured passwords where you should be using plain passwords, or vice versa:
Did you create the first config (data-drive-crypt:) using the interactive rclone config or wim?
Do the passwords you see when doing rclone config show data-drive-crypt match the passwords you have on file?
Did you create the second config (data-drive-crypt2:) using the interactive rclone config or wim?
Do the passwords you see when doing rclone config show data-drive-crypt2 match the passwords you have on file?
Forgive me. I didn't mean to imply you were the driver I just wanted you to have a better view.
Remember I am on a different server than the one that went down. So 'rclone config show ....' must go with the flag --config=rclone....conf file that I've created anew.
Yes each one I created with vim. Because the server I am restoring to also has a valid rclone/backblaze backup so if I created it with rclone config it would overwrite this servers /root/.config/rclone/rclone.conf file.
So the results look like this:
'''
[data-drive-crypt]
type = crypt
password = *** ENCRYPTED ***
password2 = *** ENCRYPTED ***
remote = data-drive:compumatter-net-svr
directory_name_encryption = false
filename_encryption = on
'''
Question, what is the purpose of the "salt" ? I kinda thought that might be a back door for unencrypting something or regenerating a way in
I feel with some degree of confidence I know the original salt and original password. Do they provide any ability to regenerate a way in?
Also, what is the purpose of password2 that I have in my config file? Where is this used?
I do and that is also what I expect you to do, so that is fine. Just keep doing that unless I tell you otherwise.
No, not if using --config=...
password2 is the salt and it is used to do the encryption, so no decryption without knowing both password1 and password2. password2 is optional, so the correct value can be "".
Does the passwords you see in the config of data-drive-crypt2 (using wim) match the passwords you have on file?
If yes, then try creating a new remote data-drive-crypt3 using rclone config --config=... were you enter the exact same settings and passwords. This should result in a config entry where the passwords are different/obfuscated from data-drive-crypt2 (when viewed in wim).
