Corrupted crypt filename

Help and Support
1

What is the problem you are having with rclone?

i just noticed a corrupted crypt filename and not sure what corrupted it?

DEBUG : ~tmp2F_e8r7nf8ge88uspuaid7912ea796pktenj1ciut56shf7pcl04ns0: Skipping undecryptable file name: illegal base32 data at input byte 0

the strange thing is that, if i remove the leading ~tmp2F_, the remainder is a valid crypt filename
e8r7nf8ge88uspuaid7912ea796pktenj1ciut56shf7pcl04ns0
and the leading text is not just some random text.
~tmp
+
2F - which is a forward slash

so what caused this to happen?

  • issue with rclone
  • issue with onedrive

note: the python script that i use to run rclone, creates a debug log.
i searched thru the logs and found this first occurrence on 20210628.165244

i uploaded that log file
C:\data\rclone\logs\keepass_files_onedrivevb_crypt\20210628.165244\rclone.log
to
https://pastebin.com/raw/tcDCD6vA

What is your rclone version (output from rclone version)

at that time of the first occurnce, `v1.55.1` on windows
as for the manual commands run in the post, using
rclone v1.56.2
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.10.60.1-microsoft-standard-WSL2 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.16.8
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

the original crypt remote was in onedrive.
for testing, i isolated the corrupt file, copied it to local and created a on-the-fly remote containing that single file.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

export cryptpath='/home/user01/rclone/crypt'
export RCLONE_CONFIG_CRYPT_TYPE=crypt
export RCLONE_CONFIG_CRYPT_REMOTE=$cryptpath
export RCLONE_CONFIG_CRYPT_PASSWORD=
export RCLONE_CONFIG_CRYPT_PASSWORD2=

rclone lsl $cryptpath -vv
rclone lsl crypt: --crypt-show-mapping -vv

The rclone config contents with secrets removed.

on the fly crypt remote.

A log from the command with the -vv flag

DEBUG : rclone: Version "v1.56.2" starting with parameters ["rclone" "lsl" "/home/user01/rclone/crypt" "-vv"]
DEBUG : Creating backend with remote "/home/user01/rclone/crypt"
DEBUG : Using config file from "/home/user01/.config/rclone/rclone.conf"
      967 2021-06-28 16:52:59.000000000 e8r7nf8ge88uspuaid7912ea796pktenj1ciut56shf7pcl04ns0
      967 2021-06-28 16:52:59.000000000 ~tmp2F_e8r7nf8ge88uspuaid7912ea796pktenj1ciut56shf7pcl04ns0

DEBUG : rclone: Version "v1.56.2" starting with parameters ["rclone" "lsl" "crypt:" "--crypt-show-mapping" "-vv"]
DEBUG : Creating backend with remote "crypt:"
DEBUG : Setting type="crypt" for "crypt" from environment variable RCLONE_CONFIG_CRYPT_TYPE
DEBUG : Setting remote="/home/user01/rclone/crypt" for "crypt" from environment variable RCLONE_CONFIG_CRYPT_REMOTE
DEBUG : Setting password="" for "crypt" from environment variable RCLONE_CONFIG_CRYPT_PASSWORD
DEBUG : Setting password2="" for "crypt" from environment variable RCLONE_CONFIG_CRYPT_PASSWORD2
DEBUG : crypt: detected overridden config - adding "{Tu7hR}" suffix to name
DEBUG : Setting remote="/home/user01/rclone/crypt" for "crypt" from environment variable RCLONE_CONFIG_CRYPT_REMOTE
DEBUG : Using config file from "/home/user01/.config/rclone/rclone.conf"
DEBUG : Setting password="" for "crypt" from environment variable RCLONE_CONFIG_CRYPT_PASSWORD
DEBUG : Setting password2="" for "crypt" from environment variable RCLONE_CONFIG_CRYPT_PASSWORD2
DEBUG : Creating backend with remote "/home/user01/rclone/crypt"
DEBUG : fs cache: renaming cache item "crypt:" to be canonical "crypt{Tu7hR}:"
      919 2021-06-28 16:52:59.000000000 KDBX_PasswordsOnly_TXT.xsl
DEBUG : ~tmp2F_e8r7nf8ge88uspuaid7912ea796pktenj1ciut56shf7pcl04ns0: Skipping undecryptable file name: illegal base32 data at input byte 0
NOTICE: KDBX_PasswordsOnly_TXT.xsl: Encrypts to "e8r7nf8ge88uspuaid7912ea796pktenj1ciut56shf7pcl04ns0"
2

You are mixing crypt and non-crypt file parts and telling crypt they are all crypt.

Chances are, you are messing up your on-the-fly crypt remote. And, given that your command is rclone lsl crypt: this is not on the fly.

3

hi,
i think you are missing the point of my post.
imho, this is not just a simple mix-up of crypted and non-crypted flies.

in that onedrive crypt there is single filename that is not correct.
and the corrupted filename is composed of two parts
~tmp2F_e8r7nf8ge88uspuaid7912ea796pktenj1ciut56shf7pcl04ns0

  • ~tmp2F_
  • e8r7nf8ge88uspuaid7912ea796pktenj1ciut56shf7pcl04ns0

note that as seen in rclone lsl crypt:
e8r7nf8ge88uspuaid7912ea796pktenj1ciut56shf7pcl04ns0 is a valid crypted filename which decrypts to KDBX_PasswordsOnly_TXT.xsl

NOTICE: KDBX_PasswordsOnly_TXT.xsl: Encrypts to "e8r7nf8ge88uspuaid7912ea796pktenj1ciut56shf7pcl04ns0"

i posted the debug log where it first occurred at pastebin.
did you take a look at it?

for testing, i copied that singe file from onedrive crypt to local, created an on-the-fly remote and ran the rclone lsl on that.

i am looking for what corrupted that filename, either rclone did it or onedrive did it.
if you think there is a third way, please let me know?

what makes you think that?
the crypt is created on the fly, using export RCLONE_CONFIG_CRYPT_TYPE=crypt

4

and in the onedrive crypt folder, here are the files
notice the first two files

rclone. lsl onedrivevb:crypt01/83jmcan84pcee32d1p7890mmlo/vplun6pmri0mr96fadebs0mm0s/ebue23o4ems817a5kdmjh5ureg/qkla6d84saeepjt9a8fphrvivk 
      967 2020-01-12 12:58:50.000000000 e8r7nf8ge88uspuaid7912ea796pktenj1ciut56shf7pcl04ns0
      967 2021-06-28 16:52:59.000000000 ~tmp2F_e8r7nf8ge88uspuaid7912ea796pktenj1ciut56shf7pcl04ns0
     3604 2020-01-12 12:58:52.000000000 en6e3721t5fonuu728b8kcg10iutu365tf4uq3s0jsqh14f39fm0
     3146 2020-01-12 12:58:50.000000000 gmrn46v72e10h0uimiverkvcnj6cfcjhstgfl3u2kjnk5utih29g
     2780 2021-01-01 08:13:44.000000000 jfbma32vqjpb11uknll3db1sso
     3148 2020-01-12 12:58:48.000000000 ur61m4c82dvhnvb0437p4ki5tievc9vvug4qn49agt7jriml5v50
5

Agree, that is a worrying observation and you should try to find the root cause.

I just scanned my crypt on SFTP and found no issues, it is created by sync’s.

I see your issue affects an Excel file. Do you use Excel on top of a mount of a crypt on top of OneDrive? (Just to understand all the variables involved)

6

Yes, I think I did miss that it was set in the environment. I am not used to that even for on-the-fly. Sorry about that.

It is odd that it is adding the ~tmp2F_ part. It would still be a bug but does it do that for different passwords (obviously not useful but would be interesting to see). What about for the same passwords, etc but done in a config file?

7

no worries,

sure this is not about crypt passwords.

in the debug log at pastebin, rclone had errors with onedrive when trying to upload that file.

DEBUG : qkla6d84saeepjt9a8fphrvivk/e8r7nf8ge88uspuaid7912ea796pktenj1ciut56shf7pcl04ns0: Cancelling multipart upload: invalidRequest: Invalid request
NOTICE: qkla6d84saeepjt9a8fphrvivk/e8r7nf8ge88uspuaid7912ea796pktenj1ciut56shf7pcl04ns0: Failed to cancel multipart upload: itemNotFound: The upload session was not found
ERROR : XSL/KDBX_PasswordsOnly_TXT.xsl: Failed to copy: invalidRequest: Invalid request
8

thanks for the confirmation.

if you get a chance, in that debug log at pastebin, there are some onedrive errors.

KDBX_PasswordsOnly_TXT.xsl.
tho the extension looks like .xls it is .xsl

for sure not, that rclone command was created by my python script rcloner.exe using this info from rcloner.ini

[keepass_files_onedrivevb_crypt]
SourceDir=c:\data\keepass
DoVShadow=true
DoRcloneSyncFiles=--stats=0 --fast-list
DoRcloneCryptCheckFiles=--stats=0 --fast-list
DoProcessLogs=true
DoSendEmailLogs=true
DestDirRclone=onedrivevb_crypt:en07
RcloneConfigFile=onedrive.conf
DoRcloneArchiveDir=true
#DoRcloneDryRun=true
#DoDebugMode=True

and creates and run these commands

C:\data\rclone\scripts\rclone.exe sync "b:\mount\rcloner\keepass_files_onedrivevb_crypt_20210628.165244\data\keepass" "onedrivevb_crypt:en07keepass/rclone/backup" --backup-dir=onedrivevb_crypt:en07keepass/rclone/archive/20210628.165244 --stats=0 --fast-list  --log-level=DEBUG --log-file=C:\data\rclone\logs\keepass_files_onedrivevb_crypt\20210628.165244\rclone.log --config=C:\data\rclone\scripts\onedrive.conf --ask-password
C:\data\rclone\scripts\rclone.exe cryptcheck "b:\mount\rcloner\keepass_files_onedrivevb_crypt_20210628.165244\data\keepass" "onedrivevb_crypt:en07keepass/rclone/backup" --stats=0 --fast-list --log-level=DEBUG --log-file=C:\data\rclone\logs\keepass_files_onedrivevb_crypt\20210628.165244\rclone.log --config=C:\data\rclone\scripts\onedrive.conf --ask-password
9

Seen this?

1 Like
10

thanks much, i did search the forum and missed that.

based on my debug log, i was thinking/hoping it was an issue with multipart uploads with onedrive.
tho i still want to confirm what was to blame, rclone, onedrive, nework glitch or what?

DEBUG : qkla6d84saeepjt9a8fphrvivk/e8r7nf8ge88uspuaid7912ea796pktenj1ciut56shf7pcl04ns0: Cancelling multipart upload: invalidRequest: Invalid request
NOTICE: qkla6d84saeepjt9a8fphrvivk/e8r7nf8ge88uspuaid7912ea796pktenj1ciut56shf7pcl04ns0: Failed to cancel multipart upload: itemNotFound: The upload session was not found
ERROR : XSL/KDBX_PasswordsOnly_TXT.xsl: Failed to copy: invalidRequest: Invalid request
11

Sure, easy to miss. It was the multipart error message that lead me to the link, via google.

So its a previous multipart upload that has been interrupted or something, leaving behind a temp file, so should be no real damage, luckily.

That is a good question, though. I don't know. My guess would be network glitch.

12

@asdffdsa,If you add --filter "- ~tmp*_" would that break any other use cases for you?

13

thanks,

before i posted, i did a scan of all the crypts i have in all different cloud providers and local crypts.
i only find that one instance i posted about.

in addition i have approx. 100,000 rclone debug log files spread over a bunch of windows machines, mostly as backup servers all running that same python script rcloner.exe.
i scanned all of them for various text such as

  • tmp - i filtered out any false positives.
  • undecryptable

in the end, i did not find a second instance.

so i think it is safe to state that this was a one-time freak glitch, that despite having a debug log of the first occurrence, i will never know exactly that happened.

tho i choose to blame micro$oft zerodrive.

14

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.