Filename changed if using copy or sync command

What is the problem you are having with rclone?

I have the exact same problem with this topic(Filename changed if using copy command). however, it was closed. I can not reply to this topic.

I also faced an extra problem which was not mentioned.

For example:
"test?.txt" is saved in onedrive and the "?" is fullwidth.

On windows platforms, using --local-encoding None or --local-encoding "Slash,LtGt,DoubleQuote,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot" can correctly solve the name changing problem of fullwidth "?".

On debian 11, it does not need --local-encoding and works well.

However, on macos platforms, the fullwidth "?" would always be changed to semiwidth "?" when using copy or sync. --local-encoding does not work on macos.

I hava plenty of files of this kind, and their filenames are all stored in sqlite. So, on macos platforms, I cannot read them because of wrong path.

As you probably already know from reading mentioned thread it is not a bug.

I understand from your post that you have no issues with Windows and Linux, only macOS.

I have just tested it and I can not reproduce it on macOS. fullwidth "?" remains always fullwidth "?".... So could you please provide an example of this not working? Include all details.

In my onedrive cloud, there is a file "onedrive:documents/others/10.重提这个问题?.txt"

On macos platforms (both 13.5.2 on M1 pro or 12.6.8 on Intel pro), after running the command rclone sync "onedrive:documents" . --checksum --progress, the filename '10.重提这个问题?.txt' became '10.重提这个问题?.txt'.(fullwidth"?" became semiwidth"?" )

If I run a test commant like rclone sync "onedrive:documents/others/10.重提这个问题?.txt" . --checksum --progress to get a specific file, roclone would print
log 2023/09/08 21:16:54 ERROR : OneDrive root 'documents/others/10.重提这个问题?.txt': error reading source root directory: directory not found 2023/09/08 21:16:54 ERROR : Local file system at /Users/......./Downloads/tmp: not deleting files as there were IO errors 2023/09/08 21:16:54 ERROR : Local file system at /Users/......./Downloads/tmp: not deleting directories as there were IO errors 2023/09/08 21:16:54 ERROR : Attempt 1/3 failed with 1 errors and: directory not found 2023/09/08 21:16:55 ERROR : OneDrive root 'documents/others/10.重提这个问题?.txt': error reading source root directory: directory not found 2023/09/08 21:16:55 ERROR : Local file system at /Users/......./Downloads/tmp: not deleting files as there were IO errors 2023/09/08 21:16:55 ERROR : Local file system at /Users/......./Downloads/tmp: not deleting directories as there were IO errors 2023/09/08 21:16:55 ERROR : Attempt 2/3 failed with 1 errors and: directory not found 2023/09/08 21:16:55 ERROR : OneDrive root 'documents/others/10.重提这个问题?.txt': error reading source root directory: directory not found 2023/09/08 21:16:55 ERROR : Local file system at /Users/......./Downloads/tmp: not deleting files as there were IO errors 2023/09/08 21:16:55 ERROR : Local file system at /Users/......./Downloads/tmp: not deleting directories as there were IO errors 2023/09/08 21:16:55 ERROR : Attempt 3/3 failed with 1 errors and: directory not found 2023/09/08 21:16:55 Failed to sync: directory not found

But when I changed the command to rclone sync "onedrive:documents/others/10.重提这个问题?.txt" . --checksum --progress (I deliberately changed the fullwith'?' to semiwidth'?'), the file '10.重提这个问题?.txt' was downloaded.

I am pretty sure the real name on onedrive is '10.重提这个问题?.txt'. I checked it again on onedive just before posting this.

Thank you for all details.

I have created two files containing "?" on my onedrive remote.

$ rclone ls onedrive:Test
        9 10.重提这个问题?.txt
        9 test?test.txt

then synced them to local macOS folder:

$ rclone sync onedrive:Test .

$ ls -l
total 16
-rw-r--r--@ 1 kptsky  staff     9B Sep  8 14:03 10.重提这个问题?.txt
-rw-r--r--@ 1 kptsky  staff     9B Sep  8 14:03 test?test.txt

I can also force sync of specific file without any issues:

$ rclone sync onedrive:Test/10.重提这个问题?.txt . -vv
2023/09/08 14:50:43 DEBUG : rclone: Version "v1.63.1" starting with parameters ["rclone" "sync" "onedrive:Test/10.重提这个问题?.txt" "." "-vv"]
2023/09/08 14:50:43 DEBUG : Creating backend with remote "onedrive:Test/10.重提这个问题?.txt"
2023/09/08 14:50:43 DEBUG : Using config file from "/Users/kptsky/.config/rclone/rclone.conf"
2023/09/08 14:50:44 DEBUG : fs cache: adding new entry for parent of "onedrive:Test/10.重提这个问题?.txt", "onedrive:Test"
2023/09/08 14:50:44 DEBUG : Creating backend with remote "."
2023/09/08 14:50:44 DEBUG : fs cache: renaming cache item "." to be canonical "/Users/kptsky/Temp/test"
2023/09/08 14:50:44 DEBUG : 10.重提这个问题?.txt: Need to transfer - File not found at Destination
2023/09/08 14:50:45 DEBUG : 10.重提这个问题?.txt: quickxor = 6120c31cc23007328441010a0900000000000000 OK
2023/09/08 14:50:45 DEBUG : 10.重提这个问题?.txt.xecazir1.partial: renamed to: 10.重提这个问题?.txt
2023/09/08 14:50:45 INFO  : 10.重提这个问题?.txt: Copied (new)
2023/09/08 14:50:45 INFO  :
Transferred:   	          9 B / 9 B, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         2.1s

2023/09/08 14:50:45 DEBUG : 11 go routines active

but when I change the full width "?" to semi width "?" it fails - which makes sense - on onedrive remote I have file with full width "?"

$ rclone sync onedrive:Test/10.重提这个问题?.txt .
2023/09/08 14:53:50 ERROR : OneDrive root 'Test/10.重提这个问题?.txt': error reading source root directory: directory not found
2023/09/08 14:53:50 ERROR : Local file system at /Users/kptsky/Temp/test: not deleting files as there were IO errors
2023/09/08 14:53:50 ERROR : Local file system at /Users/kptsky/Temp/test: not deleting directories as there were IO errors
2023/09/08 14:53:50 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
...
2023/09/08 14:53:50 Failed to sync: directory not found

Now I am wondering what is different between our systems...

BTW in your rclone.conf file don't you have "local" remote with custom encoding defined?

OK - this is how it works for me - now I created also file with semi width "?":

$ rclone ls onedrive:Test
        9 10.重提这个问题?.txt
        0 test?test.txt
        0 test?test.txt

So something is converting on your system "?" to "?"...

Yeah, I am quite wondering where😂?I installed rclone by brew, and I did not set rclone because I just copied the rclone.conf from windows to ~/.conf/rclone/

can you run on your files:

rclone ls onedrive:

and

rclone ls onedrive: --local-encoding None

and

rclone ls onedrive: --local-encoding None --onedrive-encoding None

Wow! --onedrive-encoding None works.

default should be:

--onedrive-encoding MultiEncoder The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot)

so I think it should be enough to only remove "Question"

--onedrive-encoding "Slash,LtGt,DoubleQuote,Colon,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot"

you could also add this to config file in onedrive section:
encoding = "Slash,LtGt,DoubleQuote,Colon,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot"

Ok! I will try. Thanks a lot and sorry for bothering you.

No problem:) Problem is characters encoding:)

Definitely. Worldwide developers are suffering from encoding.