Possible to create second crypt remote that emulates cache crypt remote?

td001 · June 7, 2019, 2:12am

I would like to have a cached remote mount AND a direct remote (for copying files to Google Drive) as per this forum topic: Rclone Crypt with Cache, can't copy files to Drive?

My current setup is: drive -> cache -> crypt

I have created a second crypt: drive -> crypt2

Now everything works as it should, however my issue is that when I upload a file via this new crypt2 it does not place the file in folders I have already created via the cached crypt. Is there a way I can do this? I only ask because via the first remote I have folders created called Movies/TV/Music and I have Sonarr and Radarr pointing to these.

Animosity022 · June 7, 2019, 2:20am

Can you share your rclone.conf without the keys and passwords?

If you are using the same crypt in both spots, it should be fine.

td001 · June 7, 2019, 2:34am

Sure can:

[gdrive]
type = drive
scope = drive
token = {"access_token":"redacted","token_type":"Bearer","refresh_token":"redacted","expiry":"2019-06-06T22:50:02.148007267-04:00"}
client_secret = redacted
client_id = redacted.apps.googleusercontent.com

[cache]
type = cache
remote = gdrive:/secure
plex_url = http://192.168.12.119:32400
plex_username = redacted@gmail.com
plex_password = redacted
chunk_size = 10M
info_age = 2d
chunk_total_size = 10G

[crypt]
type = crypt
remote = cache:/crypt
filename_encryption = standard
directory_name_encryption = true
password = redacted
password2 = redacted

[crypt2]
type = crypt
remote = gdrive:/secure/crypt
filename_encryption = standard
directory_name_encryption = true
password = redacted
password2 = redacted

Animosity022 · June 7, 2019, 2:53am

What's the command you are running that doesn't work right?

td001 · June 7, 2019, 2:55am

if pidof -o %PPID -x "gdrive-upload-movies.sh"; then
echo "Script is already running. Terminating."
exit 1
fi

LOGFILE="/home/td001/logs/gdrive-upload-movies.log"
FROM="/home/td001/TorrentShare/RadarrDownloads"
#TO="/home/td001/mnt/gdrive/Movies/"
TO="crypt2:/Movies"
CONFIG="/home/td001/.config/rclone/rclone.conf"

echo "$(date "+%d.%m.%Y %T") RCLONE UPLOAD STARTED" | tee -a $LOGFILE
# MOVE FILES TO GOOGLE DRIVE
rclone move "$FROM" "$TO" --config=$CONFIG --log-file=$LOGFILE -vv --transfers=10 --checkers=3 --no-traverse --tpslimit 3 --drive-chunk-size 32M --ignore-existing --exclude *partial~ --delete-after
echo "$(date +"%D %T") RCLONE UPLOAD FINISHED" | tee -a $LOGFILE

The command works, it's just not saving to the correct location on Google Drive. It will create it's own folder instead of inserting into the Movies folder that already existed when I created it through the cache mount on my system (I included the location it's the TO that's commented out).

Animosity022 · June 7, 2019, 3:15am

Can you run the command without the script and just copy something up and include the log?

td001 · June 7, 2019, 3:55am

For sure:

rclone move --config=/home/td001/.config/rclone/rclone.conf -vv /home/td001/TorrentShare/RadarrDownloads crypt2:/Movies --log-file=/home/td001/TorrentShare/testtransfer.log --transfers=10 --checkers=3 --no-traverse --tpslimit 3 --drive-chunk-size 32M --ignore-existing --exclude *partial~ --delete-after --dry-run

2019/06/06 23:46:38 DEBUG : Using config file from "/home/td001/.config/rclone/rclone.conf"
2019/06/06 23:46:38 INFO  : Starting HTTP transaction limiter: max 3 transactions/s with burst 1
2019/06/06 23:46:40 INFO  : Encrypted drive 'crypt2:/Movies': Waiting for checks to finish
2019/06/06 23:46:40 INFO  : Encrypted drive 'crypt2:/Movies': Waiting for transfers to finish
2019/06/06 23:46:40 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 0, -
Elapsed time:        2.3s

2019/06/06 23:46:40 DEBUG : 4 go routines active
2019/06/06 23:46:40 DEBUG : rclone: Version "v1.47.0" finishing with parameters ["rclone" "move" "--config=/home/td001/.config/rclone/rclone.conf" "-vv" "/home/td001/TorrentShare/RadarrDownloads" "crypt2:/Movies" "--log-file=/home/td001/TorrentShare/testtransfer.log" "--transfers=10" "--checkers=3" "--no-traverse" "--tpslimit" "3" "--drive-chunk-size" "32M" "--ignore-existing" "--exclude" "*partial~" "--delete-after" "--dry-run"]

Run successfully - however, still created it's own folder within 'crypt' next to the other folders: Movies, TV, Music instead of being placed in the Movies folder. I'm sure I'm missing some tiny silly config somewhere though I appreciate your help @Animosity022 you're like legendary around here

Animosity022 · June 7, 2019, 4:00am

The cache or non cache should point to the same thing if I'm reading your config right.

It looks like they are both pointing to gdrive:/secure/crypt

Can you just one file up?

rclone copy test.txt crypt2:/Movies -vv

td001 · June 7, 2019, 4:07am

Yep:

2019/06/07 00:04:26 DEBUG : rclone: Version "v1.47.0" starting with parameters ["rclone" "copy" "test.txt" "crypt2:/Movies" "-vv"]
2019/06/07 00:04:26 DEBUG : Using config file from "/home/td001/.config/rclone/rclone.conf"
2019/06/07 00:04:29 DEBUG : test.txt: Couldn't find file - need to transfer
2019/06/07 00:04:32 INFO  : test.txt: Copied (new)
2019/06/07 00:04:32 INFO  :
Transferred:            32 / 32 Bytes, 100%, 5 Bytes/s, ETA 0s
Errors:                 0
Checks:                 0 / 0, -
Transferred:            1 / 1, 100%
Elapsed time:        5.3s

2019/06/07 00:04:32 DEBUG : 4 go routines active
2019/06/07 00:04:32 DEBUG : rclone: Version "v1.47.0" finishing with parameters ["rclone" "copy" "test.txt" "crypt2:/Movies" "-vv"]

Successful - however again it seems to create a whole new folder instead of copying it into the existing Movie folder.

Animosity022 · June 7, 2019, 4:10am

What does rclone lsd crypt2: -vv show?

td001 · June 7, 2019, 4:16am

~$ rclone lsd crypt2: -vv
2019/06/07 00:14:47 DEBUG : rclone: Version "v1.47.0" starting with parameters ["rclone" "lsd" "crypt2:" "-vv"]
2019/06/07 00:14:47 DEBUG : Using config file from "/home/td001/.config/rclone/rclone.conf"
2019/06/07 00:14:48 DEBUG : h5i2jsjh5hqq9c2jpa4q6ip9l0: Skipping undecryptable dir name: Bad PKCS#7 padding - too long
2019/06/07 00:14:48 DEBUG : br4l7oppte4ln8ma74vj43u730: Skipping undecryptable dir name: Bad PKCS#7 padding - too long
2019/06/07 00:14:48 DEBUG : gu429mg4onhpaq5g5jm100419o: Skipping undecryptable dir name: Bad PKCS#7 padding - too long
          -1 2019-06-07 00:14:39        -1 Movies
2019/06/07 00:14:48 DEBUG : 4 go routines active
2019/06/07 00:14:48 DEBUG : rclone: Version "v1.47.0" finishing with parameters ["rclone" "lsd" "crypt2:" "-vv"]

Animosity022 · June 7, 2019, 4:22am

Are the passwords both the same for the crypt1 and crypt2? That error usually comes from a bad salt / bad password.

td001 · June 7, 2019, 4:26am

YES! The passwords are both the same but the salts are different.

Animosity022 · June 7, 2019, 4:32am

I believe that would create the error you are seeing. You'd want to copy the passwords from the first section and use the same in the second crypt.

I recreated it myself with the same password and a different salt.

[felix@gemini rclone]$ rclone lsd test: -vv
2019/06/07 00:30:09 DEBUG : rclone: Version "v1.47.0" starting with parameters ["rclone" "lsd" "test:" "-vv"]
2019/06/07 00:30:09 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2019/06/07 00:30:09 DEBUG : rpju5hpe3gjgr6j3jt5jtibqvc: Skipping undecryptable file name: Bad PKCS#7 padding - too long
2019/06/07 00:30:09 DEBUG : 95tj3q4gj5ban13ppu0kisguco: Skipping undecryptable file name: Bad PKCS#7 padding - too long
2019/06/07 00:30:09 DEBUG : tnvepu36qiohcun8v84ddhsam0: Skipping undecryptable dir name: Bad PKCS#7 padding - too long
2019/06/07 00:30:09 DEBUG : smu5ej34ujbdoip1cm3mlk92q4: Skipping undecryptable dir name: Bad PKCS#7 padding - too long
2019/06/07 00:30:09 DEBUG : 4 go routines active
2019/06/07 00:30:09 DEBUG : rclone: Version "v1.47.0" finishing with parameters ["rclone" "lsd" "test:" "-vv"]
[felix@gemini rclone]$ vi rclone.conf

td001 · June 7, 2019, 4:42am

So same passwords and same salt?

Animosity022 · June 7, 2019, 10:30am

You can just copy and paste the first section of crypt completely and just change the remote location / name.

td001 · June 7, 2019, 1:16pm

Yep... It looks like this might have corrected the errors. I will try to upload again today and post results. Thanks for the help so far @Animosity022

rclone lsd crypt2: -vv
2019/06/07 09:15:03 DEBUG : rclone: Version "v1.47.0" starting with parameters ["rclone" "lsd" "crypt2:" "-vv"]
2019/06/07 09:15:03 DEBUG : Using config file from "/home/td001/.config/rclone/rclone.conf"
          -1 2019-04-27 10:58:26        -1 Movies
          -1 2019-04-27 10:59:08        -1 Music
          -1 2019-04-27 10:58:47        -1 TV
2019/06/07 09:15:05 DEBUG : 4 go routines active
2019/06/07 09:15:05 DEBUG : rclone: Version "v1.47.0" finishing with parameters ["rclone" "lsd" "crypt2:" "-vv"]

Animosity022 · June 7, 2019, 1:21pm

Cool!

You'd have to clean up the directories/files loaded with the different key or that'll toss errors. You'd not see those in your mount as it kind of hides it. The rclone ls command with -vv would show you.

td001 · June 7, 2019, 8:20pm

Confirmed working! Awesome thanks for all the help @Animosity022!

system · June 10, 2019, 8:25pm

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.