Problem with beta and special character

gulp79 · February 2, 2021, 2:27pm

What is the problem you are having with rclone?

Only using BETA, i've problem with special character (in this case with % ) in file/directory names, tested with ONEDRIVE

What is your rclone version (output from `rclone version`)

rclone: Version "v1.54.0-beta.5137.8b41dfa50"

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows10 64bit - Linux 64 Bit

Which cloud storage system are you using? (eg Google Drive)

Onedrive for business

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

With current stable release:

rclone.exe ls onecrypt:"/Portable/TYPSoft FTP Server/" -vv

With current BETA (v1.54.0-beta.5137.8b41dfa50) release:

rclone_beta.exe ls onecrypt:"/Portable/TYPSoft FTP Server/" -vv

A log from the command with the `-vv` flag

Output of stable version:

2021/02/02 15:18:46 DEBUG : rclone: Version "v1.53.4" starting with parameters ["rclone.exe" "ls" "onecrypt:/Portable/TYPSoft FTP Server/" "-vv"]
2021/02/02 15:18:46 DEBUG : Using config file from "C:\\RcloneTest\\rclone.conf"
2021/02/02 15:18:46 DEBUG : Creating backend with remote "onecrypt:/Portable/TYPSoft FTP Server/"
2021/02/02 15:18:46 DEBUG : Creating backend with remote "onedrive:/Crypt/57.WvyAhisl/58.bgXawnB NbX amzDmz"
2021/02/02 15:18:48 DEBUG : fs cache: renaming cache item "onedrive:/Crypt/57.WvyAhisl/58.bgXawnB NbX amzDmz" to be canonical "onedrive:Crypt/57.WvyAhisl/58.bgXawnB NbX amzDmz"
2021/02/02 15:18:48 DEBUG : fs cache: switching user supplied name "onedrive:/Crypt/57.WvyAhisl/58.bgXawnB NbX amzDmz" for canonical name "onedrive:Crypt/57.WvyAhisl/58.bgXawnB NbX amzDmz"
  2097152 TYPSoft FTP Server.exe
     4096 TYPSoft FTP Server/Registry.rw.tvr
       60 TYPSoft FTP Server/Registry.rw.tvr.lck
     4096 TYPSoft FTP Server/Registry.rw.tvr.transact
     2560 TYPSoft FTP Server/Registry.tlog
    32768 TYPSoft FTP Server/Registry.tlog.cache
      376 TYPSoft FTP Server/SKEL/30327f9ebdf20b78219cc8c50d2e221f26df0708.Tls
   129024 TYPSoft FTP Server/SKEL/30f0e44f4038720bb338ca7cc32fea1e308afd3e.SharedTA
      281 TYPSoft FTP Server/%drive_C%/TYPSoft FTP Server/config.ini
      312 TYPSoft FTP Server/%drive_C%/TYPSoft FTP Server/users.ini
      136 TYPSoft FTP Server/%AppData%/Dropbox/shellext/l/4fd84945
2021/02/02 15:18:50 DEBUG : 4 go routines active

Output of BETA version with error on special character:

2021/02/02 15:18:54 DEBUG : rclone: Version "v1.54.0-beta.5137.8b41dfa50" starting with parameters ["rclone_beta.exe" "ls" "onecrypt:/Portable/TYPSoft FTP Server/" "-vv"]
2021/02/02 15:18:54 DEBUG : Using config file from "C:\\RcloneTest\\rclone.conf"
2021/02/02 15:18:54 DEBUG : Creating backend with remote "onecrypt:/Portable/TYPSoft FTP Server/"
2021/02/02 15:18:54 DEBUG : Creating backend with remote "onedrive:/Crypt/57.WvyAhisl/58.bgXawnB NbX amzDmz"
2021/02/02 15:18:56 DEBUG : fs cache: renaming cache item "onedrive:/Crypt/57.WvyAhisl/58.bgXawnB NbX amzDmz" to be canonical "onedrive:Crypt/57.WvyAhisl/58.bgXawnB NbX amzDmz"
2021/02/02 15:18:56 DEBUG : fs cache: switching user supplied name "onedrive:/Crypt/57.WvyAhisl/58.bgXawnB NbX amzDmz" for canonical name "onedrive:Crypt/57.WvyAhisl/58.bgXawnB NbX amzDmz"
  2097152 TYPSoft FTP Server.exe
     4096 TYPSoft FTP Server/Registry.rw.tvr
       60 TYPSoft FTP Server/Registry.rw.tvr.lck
     4096 TYPSoft FTP Server/Registry.rw.tvr.transact
     2560 TYPSoft FTP Server/Registry.tlog
    32768 TYPSoft FTP Server/Registry.tlog.cache
2021/02/02 15:18:56 ERROR : TYPSoft FTP Server/ﾷAppDataﾷ: error listing: directory not found
2021/02/02 15:18:57 ERROR : TYPSoft FTP Server/ﾘdrive_Cﾘ: error listing: directory not found
      376 TYPSoft FTP Server/SKEL/30327f9ebdf20b78219cc8c50d2e221f26df0708.Tls
   129024 TYPSoft FTP Server/SKEL/30f0e44f4038720bb338ca7cc32fea1e308afd3e.SharedTA
2021/02/02 15:18:57 DEBUG : 4 go routines active
2021/02/02 15:18:57 Failed to ls with 3 errors: last error was: directory not found

ncw · February 2, 2021, 6:00pm

I think this is because of this commit

This makes % into a legal name for onedrive which it is.

Previously rclone will have been changing % into a unicode equivalent

I'm not sure why the ls is failing though - that is very strange

Can you try this flag which should behave the same as 1.53

--onedrive-encoding Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot,HashPercent

If that works you can put it in the config as

encoding = Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot,HashPercent

I think this problem is because you are wrapping the onedrive in a crypt - can you post your config with secrets removed please?

gulp79 · February 2, 2021, 9:25pm

Thank you for your support!!

So my config is this:

[onedrive]
type = onedrive
client_id = ***
client_secret = ***
token = ***
drive_id = ***
drive_type = business
server_side_across_configs = true

[onecrypt]
type = crypt
remote = onedrive:/Crypt
filename_encryption = obfuscate
directory_name_encryption = true
server_side_across_configs = true
password = ***

if i put flags you suggested, i've an error on

encoder.MultiEncoder failed: bad encoding "HashPercent"

all output is:

rclone.exe --onedrive-encoding Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot,HashPercent ls onecrypt:"/Portable/TYPSoft FTP Server/" -vv
Error: invalid argument "Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot,HashPercent" for "--onedrive-encoding" flag: parsing "Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot,HashPercent" as encoder.MultiEncoder failed: bad encoding "HashPercent": possible values are: Asterisk, BackQuote, BackSlash, Colon, CrLf, Ctl, Del, Dollar, Dot, DoubleQuote, Hash, InvalidUtf8, LeftCrLfHtVt, LeftPeriod, LeftSpace, LeftTilde, LtGt, None, Percent, Pipe, Question, RightCrLfHtVt, RightPeriod, RightSpace, SingleQuote, Slash
Usage:
  rclone ls remote:path [flags]

Flags:
  -h, --help   help for ls

Use "rclone [command] --help" for more information about a command.
Use "rclone help flags" for to see the global flags.
Use "rclone help backends" for a list of supported services.

2021/02/02 22:17:30 Fatal error: invalid argument "Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot,HashPercent" for "--onedrive-encoding" flag: parsing "Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot,HashPercent" as encoder.MultiEncoder failed: bad encoding "HashPercent": possible values are: Asterisk, BackQuote, BackSlash, Colon, CrLf, Ctl, Del, Dollar, Dot, DoubleQuote, Hash, InvalidUtf8, LeftCrLfHtVt, LeftPeriod, LeftSpace, LeftTilde, LtGt, None, Percent, Pipe, Question, RightCrLfHtVt, RightPeriod, RightSpace, SingleQuote, Slash

`

Seems that "HashPercent" is not a valid flag....

x0b · February 2, 2021, 9:30pm

Just a comma between Hash and Percent missing, use:
--onedrive-encoding Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot,Hash,Percent

gulp79 · February 2, 2021, 9:33pm

Thank you,
with comma and flags, now v 1.54 is working:

C:\rclonetests>rclone.exe --onedrive-encoding Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot,Hash,Percent ls onecrypt:"/Portable/TYPSoft FTP Server/" -vv

Ok, i'll add flags to config, but i think this is a bug.

2021/02/02 22:31:50 DEBUG : rclone: Version "v1.54.0" starting with parameters ["rclone.exe" "--onedrive-encoding" "Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot,Hash,Percent" "ls" "onecrypt:/Portable/TYPSoft FTP Server/" "-vv"]
2021/02/02 22:31:50 DEBUG : Using config file from "C:\\rclonetests\\rclone.conf"
2021/02/02 22:31:50 DEBUG : Creating backend with remote "onecrypt:/Portable/TYPSoft FTP Server/"
2021/02/02 22:31:50 DEBUG : Creating backend with remote "onedrive:/Crypt/57.WvyAhisl/58.bgXawnB NbX amzDmz"
2021/02/02 22:31:51 DEBUG : fs cache: renaming cache item "onedrive:/Crypt/57.WvyAhisl/58.bgXawnB NbX amzDmz" to be canonical "onedrive:Crypt/57.WvyAhisl/58.bgXawnB NbX amzDmz"
2021/02/02 22:31:51 DEBUG : fs cache: switching user supplied name "onedrive:/Crypt/57.WvyAhisl/58.bgXawnB NbX amzDmz" for canonical name "onedrive:Crypt/57.WvyAhisl/58.bgXawnB NbX amzDmz"
  2097152 TYPSoft FTP Server.exe
     4096 TYPSoft FTP Server/Registry.rw.tvr
       60 TYPSoft FTP Server/Registry.rw.tvr.lck
     4096 TYPSoft FTP Server/Registry.rw.tvr.transact
     2560 TYPSoft FTP Server/Registry.tlog
    32768 TYPSoft FTP Server/Registry.tlog.cache
      376 TYPSoft FTP Server/SKEL/30327f9ebdf20b78219cc8c50d2e221f26df0708.Tls
   129024 TYPSoft FTP Server/SKEL/30f0e44f4038720bb338ca7cc32fea1e308afd3e.SharedTA
      281 TYPSoft FTP Server/%drive_C%/TYPSoft FTP Server/config.ini
      312 TYPSoft FTP Server/%drive_C%/TYPSoft FTP Server/users.ini
      136 TYPSoft FTP Server/%AppData%/Dropbox/shellext/l/4fd84945
2021/02/02 22:31:52 DEBUG : 9 go routines active

ncw · February 3, 2021, 11:43am

I think specifically this is a bad interaction with the crypt wrapping and filename_encryption = obfuscate.

You can also fix this by re-uploading the files with# and % I think.

system · April 5, 2021, 7:43am

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.