Filename changed if using copy or sync command

Help and Support
1

What is the problem you are having with rclone?

I have the exact same problem with this topic(Filename changed if using copy command). however, it was closed. I can not reply to this topic.

I also faced an extra problem which was not mentioned.

For example:
"test？.txt" is saved in onedrive and the "？" is fullwidth.

On windows platforms, using --local-encoding None or --local-encoding "Slash,LtGt,DoubleQuote,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot" can correctly solve the name changing problem of fullwidth "？".

On debian 11, it does not need --local-encoding and works well.

However, on macos platforms, the fullwidth "？" would always be changed to semiwidth "?" when using copy or sync. --local-encoding does not work on macos.

I hava plenty of files of this kind, and their filenames are all stored in sqlite. So, on macos platforms, I cannot read them because of wrong path.

2

As you probably already know from reading mentioned thread it is not a bug.

I understand from your post that you have no issues with Windows and Linux, only macOS.

I have just tested it and I can not reproduce it on macOS. fullwidth "？" remains always fullwidth "？".... So could you please provide an example of this not working? Include all details.

3

In my onedrive cloud, there is a file "onedrive:documents/others/10.重提这个问题？.txt"

On macos platforms (both 13.5.2 on M1 pro or 12.6.8 on Intel pro), after running the command rclone sync "onedrive:documents" . --checksum --progress, the filename '10.重提这个问题？.txt' became '10.重提这个问题?.txt'.(fullwidth"？" became semiwidth"?" )

If I run a test commant like rclone sync "onedrive:documents/others/10.重提这个问题？.txt" . --checksum --progress to get a specific file, roclone would print
log 2023/09/08 21:16:54 ERROR : OneDrive root 'documents/others/10.重提这个问题？.txt': error reading source root directory: directory not found 2023/09/08 21:16:54 ERROR : Local file system at /Users/......./Downloads/tmp: not deleting files as there were IO errors 2023/09/08 21:16:54 ERROR : Local file system at /Users/......./Downloads/tmp: not deleting directories as there were IO errors 2023/09/08 21:16:54 ERROR : Attempt 1/3 failed with 1 errors and: directory not found 2023/09/08 21:16:55 ERROR : OneDrive root 'documents/others/10.重提这个问题？.txt': error reading source root directory: directory not found 2023/09/08 21:16:55 ERROR : Local file system at /Users/......./Downloads/tmp: not deleting files as there were IO errors 2023/09/08 21:16:55 ERROR : Local file system at /Users/......./Downloads/tmp: not deleting directories as there were IO errors 2023/09/08 21:16:55 ERROR : Attempt 2/3 failed with 1 errors and: directory not found 2023/09/08 21:16:55 ERROR : OneDrive root 'documents/others/10.重提这个问题？.txt': error reading source root directory: directory not found 2023/09/08 21:16:55 ERROR : Local file system at /Users/......./Downloads/tmp: not deleting files as there were IO errors 2023/09/08 21:16:55 ERROR : Local file system at /Users/......./Downloads/tmp: not deleting directories as there were IO errors 2023/09/08 21:16:55 ERROR : Attempt 3/3 failed with 1 errors and: directory not found 2023/09/08 21:16:55 Failed to sync: directory not found

But when I changed the command to rclone sync "onedrive:documents/others/10.重提这个问题?.txt" . --checksum --progress (I deliberately changed the fullwith'？' to semiwidth'?'), the file '10.重提这个问题?.txt' was downloaded.

I am pretty sure the real name on onedrive is '10.重提这个问题？.txt'. I checked it again on onedive just before posting this.

4

Thank you for all details.

I have created two files containing "？" on my onedrive remote.

$ rclone ls onedrive:Test
        9 10.重提这个问题？.txt
        9 test？test.txt

then synced them to local macOS folder:

$ rclone sync onedrive:Test .

$ ls -l
total 16
-rw-r--r--@ 1 kptsky  staff     9B Sep  8 14:03 10.重提这个问题？.txt
-rw-r--r--@ 1 kptsky  staff     9B Sep  8 14:03 test？test.txt

I can also force sync of specific file without any issues:

$ rclone sync onedrive:Test/10.重提这个问题？.txt . -vv
2023/09/08 14:50:43 DEBUG : rclone: Version "v1.63.1" starting with parameters ["rclone" "sync" "onedrive:Test/10.重提这个问题？.txt" "." "-vv"]
2023/09/08 14:50:43 DEBUG : Creating backend with remote "onedrive:Test/10.重提这个问题？.txt"
2023/09/08 14:50:43 DEBUG : Using config file from "/Users/kptsky/.config/rclone/rclone.conf"
2023/09/08 14:50:44 DEBUG : fs cache: adding new entry for parent of "onedrive:Test/10.重提这个问题？.txt", "onedrive:Test"
2023/09/08 14:50:44 DEBUG : Creating backend with remote "."
2023/09/08 14:50:44 DEBUG : fs cache: renaming cache item "." to be canonical "/Users/kptsky/Temp/test"
2023/09/08 14:50:44 DEBUG : 10.重提这个问题？.txt: Need to transfer - File not found at Destination
2023/09/08 14:50:45 DEBUG : 10.重提这个问题？.txt: quickxor = 6120c31cc23007328441010a0900000000000000 OK
2023/09/08 14:50:45 DEBUG : 10.重提这个问题？.txt.xecazir1.partial: renamed to: 10.重提这个问题？.txt
2023/09/08 14:50:45 INFO  : 10.重提这个问题？.txt: Copied (new)
2023/09/08 14:50:45 INFO  :
Transferred:   	          9 B / 9 B, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         2.1s

2023/09/08 14:50:45 DEBUG : 11 go routines active

but when I change the full width "？" to semi width "?" it fails - which makes sense - on onedrive remote I have file with full width "？"

$ rclone sync onedrive:Test/10.重提这个问题?.txt .
2023/09/08 14:53:50 ERROR : OneDrive root 'Test/10.重提这个问题?.txt': error reading source root directory: directory not found
2023/09/08 14:53:50 ERROR : Local file system at /Users/kptsky/Temp/test: not deleting files as there were IO errors
2023/09/08 14:53:50 ERROR : Local file system at /Users/kptsky/Temp/test: not deleting directories as there were IO errors
2023/09/08 14:53:50 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
...
2023/09/08 14:53:50 Failed to sync: directory not found

Now I am wondering what is different between our systems...

5

BTW in your rclone.conf file don't you have "local" remote with custom encoding defined?

7

OK - this is how it works for me - now I created also file with semi width "?":

image
image1676×500 19 KB

$ rclone ls onedrive:Test
        9 10.重提这个问题？.txt
        0 test?test.txt
        0 test？test.txt

So something is converting on your system "？" to "?"...

8

Yeah, I am quite wondering where😂？I installed rclone by brew, and I did not set rclone because I just copied the rclone.conf from windows to ~/.conf/rclone/

9

can you run on your files:

rclone ls onedrive:

and

rclone ls onedrive: --local-encoding None

and

rclone ls onedrive: --local-encoding None --onedrive-encoding None

10

Wow! --onedrive-encoding None works.

image
image2050×564 114 KB

11

default should be:

--onedrive-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot)

so I think it should be enough to only remove "Question"

--onedrive-encoding "Slash,LtGt,DoubleQuote,Colon,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot"

you could also add this to config file in onedrive section:
encoding = "Slash,LtGt,DoubleQuote,Colon,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot"

12

Ok! I will try. Thanks a lot and sorry for bothering you.

13

No problem:) Problem is characters encoding:)

14

Definitely. Worldwide developers are suffering from encoding.