Rclone Crypt Dropbox

So then impossible to make this operation server side with the last beta version I guess...

Anyways I will try renaming with rclone moveto and trying to manage that. Althought it would be very very nice if I could do this rclone move from unencrypted to encrypted remote server side.

You still can do it server side, it's just a lot more scripted / annoying unless @ncw has some fix / other thing I'm not getting.

You can move the file on the non encrypted remote server side with the proper name.

So in my example, I'd move 'hosts' to 'k072gvbe6gitl3j2u47snvg7c8' as that's the proper name for the crypt config / passwords I've setup and that file in the crypted remote would work properly.

Now, mapping that all out for every file is a bit annoying but depends on what your goal is and how much data.

You can probably get 1TB every 2 hours on something like the VPS I shared so depending on how much, letting it run might be an option. If you have 100s of TB, that's probably not the way to go.

Yeah thats the thing... I have petabytes of files... so what I think I will do is generating lots of 0 bytes files named "randomname".plot and then upload them to the encrypted remote and extract all names. Its a bit difficult with so many files and I might get some limits/issues. The thing is that the filename is not important, as soon as the file its ended in .plot, it will be detected and mapped correctly. Any Idea of how to encrypt the names of many in an easy way?

For example, encrypting a list like this:

akjgefjkaghfkhrfk.plot
akjefhkjaehfkahfka.plot
akjefhkajhfkaea.plot
320111036013103.plot

a list like this with thousands of names (as I say no matter the name, just the extension .plot).

Once I got that list of thousands of randomtext.plot encrypted, then I could start making my rclone moveto scripts. Thats the idea.

The extensions don't matter.

So in my example for one file I grabbed the name by copying it to the encrypted remote I setup:

felix@gemini:~$ rclone copyto DB:testme.txt TESTDBC:testme.txt -vvv
2022/08/04 11:49:58 DEBUG : Setting --config "/opt/rclone/rclone.conf" from environment variable RCLONE_CONFIG="/opt/rclone/rclone.conf"
2022/08/04 11:49:58 DEBUG : rclone: Version "v1.59.0" starting with parameters ["rclone" "copyto" "DB:testme.txt" "TESTDBC:testme.txt" "-vvv"]
2022/08/04 11:49:58 DEBUG : Creating backend with remote "DB:testme.txt"
2022/08/04 11:49:58 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2022/08/04 11:49:58 DEBUG : fs cache: adding new entry for parent of "DB:testme.txt", "DB:"
2022/08/04 11:49:58 DEBUG : Creating backend with remote "TESTDBC:"
2022/08/04 11:49:58 DEBUG : Creating backend with remote "DB:testing"
2022/08/04 11:49:59 DEBUG : testme.txt: Need to transfer - File not found at Destination
2022/08/04 11:50:00 DEBUG : tpjemqq4k8ef1qu0fhslhtnsqk: Uploading chunk 1/1
2022/08/04 11:50:01 DEBUG : tpjemqq4k8ef1qu0fhslhtnsqk: Uploading chunk 2/1
2022/08/04 11:50:02 DEBUG : Dropbox root 'testing': Adding "/testing/tpjemqq4k8ef1qu0fhslhtnsqk" to batch
2022/08/04 11:50:02 DEBUG : Dropbox root 'testing': Batch idle for 500ms so committing
2022/08/04 11:50:02 DEBUG : Dropbox root 'testing': Committing sync batch length 1 starting with: /testing/tpjemqq4k8ef1qu0fhslhtnsqk
2022/08/04 11:50:02 DEBUG : Dropbox root 'testing': Committed sync batch length 1 starting with: /testing/tpjemqq4k8ef1qu0fhslhtnsqk
2022/08/04 11:50:02 INFO  : testme.txt: Copied (new)
2022/08/04 11:50:02 INFO  :
Transferred:   	        327 B / 327 B, 100%, 108 B/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:         4.6s

2022/08/04 11:50:02 DEBUG : 12 go routines active
2022/08/04 11:50:02 INFO  : Dropbox root '': Commiting uploads - please wait...
2022/08/04 11:50:02 INFO  : Dropbox root 'testing': Commiting uploads - please wait...

I know the name now so I deleted the file at the destination crypt.

efelix@gemini:~$ rclone moveto DB:testme.txt DB:testing/tpjemqq4k8ef1qu0fhslhtnsqk -vvv
2022/08/04 11:51:23 DEBUG : Setting --config "/opt/rclone/rclone.conf" from environment variable RCLONE_CONFIG="/opt/rclone/rclone.conf"
2022/08/04 11:51:23 DEBUG : rclone: Version "v1.59.0" starting with parameters ["rclone" "moveto" "DB:testme.txt" "DB:testing/tpjemqq4k8ef1qu0fhslhtnsqk" "-vvv"]
2022/08/04 11:51:23 DEBUG : Creating backend with remote "DB:testme.txt"
2022/08/04 11:51:23 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2022/08/04 11:51:23 DEBUG : fs cache: adding new entry for parent of "DB:testme.txt", "DB:"
2022/08/04 11:51:23 DEBUG : Creating backend with remote "DB:testing/"
2022/08/04 11:51:23 DEBUG : fs cache: renaming cache item "DB:testing/" to be canonical "DB:testing"
2022/08/04 11:51:23 DEBUG : testme.txt: Need to transfer - File not found at Destination
2022/08/04 11:51:25 INFO  : testme.txt: Moved (server-side) to: tpjemqq4k8ef1qu0fhslhtnsqk
2022/08/04 11:51:25 INFO  :
Transferred:   	        327 B / 327 B, 100%, 0 B/s, ETA -
Checks:                 1 / 1, 100%
Renamed:                1
Elapsed time:         1.6s

2022/08/04 11:51:25 DEBUG : 7 go routines active
2022/08/04 11:51:25 INFO  : Dropbox root '': Commiting uploads - please wait...
2022/08/04 11:51:25 INFO  : Dropbox root 'testing': Commiting uploads - please wait...

and file translates properly.

felix@gemini:~$ rclone lsf TESTDBC:
testme.txt

The reason I suggested a local remote is you can use 0 byte files to get a full listing of the files crypt names and you'd have to script the moves out.

Ok I will try then using a local encrypted remote and adding there thousands of files with randomname.plot. Then I will take those names encrypted and generate my rclone moveto script.

Please please, test with one file, validate it works.

Try a few more, validate.

Test with a larger number, validate.

You have a lot of data you are talking about and moves are a semi destructive act so I may not have something perfect as I can't see exactly everything on your side.

Ok!! I will make this slowly and carefully. Thank you very much.

You are right --server-side-across-configs doesn't work from non-crypt to crypt. It does work from crypt to crypt though, so if you wanted to change your file name encryption it would work for example.

As others say, you must download and re-upload all content you want encrypted. There are no shortcuts around that. I am in the same boat right now. I'm downloading over 500TB of unencrypted files from my GDrive and re-uploading them using rclone crypt on a new GDrive account. It will take around 7 months to complete, I'm half way there. I rented a dedicated storage server from Hetzner for this -- I highly recommend Hetzner. I'm using their SX64 dedi. Good luck!

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.