Unicode characters needlessly escaped when syncing to sftp remote

asdffdsa · November 6, 2025, 4:16pm

still not able to reproduce your issue.

here is another test.
src=windows ntfs filesystem on internal ssd.
dst=external usb drive formatted as ntfs. that is connected to a synology device. accssed over tailscale

rclone copy d:\zork bnas_sftp_ts:usbshare1-2/zork -vv --inplace --retries=1 
DEBUG : rclone: Version "v1.71.2" starting with parameters ["d:\\data\\rclone\\rclone.exe" "copy" "d:\\zork" "bnas_sftp_ts:usbshare1-2/zork" "-vv" "--inplace" "--retries=1"]
DEBUG : Creating backend with remote "d:\\zork"
DEBUG : Using config file from "d:\\data\\rclone\\rclone.conf"
DEBUG : fs cache: renaming cache item "d:\\zork" to be canonical "//?/d:/zork"
DEBUG : Creating backend with remote "bnas_sftp_ts:usbshare1-2/zork"
DEBUG : sftp://user99@bnas_ts:22/usbshare1-2/zork: New connection 100.109.123.23:22465->100.115.37.34:22 to "SSH-2.0-OpenSSH_8.2"
DEBUG : sftp://user99@bnas_ts:22/usbshare1-2/zork: Shell type "unix" from config
DEBUG : sftp://user99@bnas_ts:22/usbshare1-2/zork: Relative path resolved to "/usbshare1-2/zork"
DEBUG : sftp://user99@bnas_ts:22/usbshare1-2/zork: Using root directory "/usbshare1-2/zork"
DEBUG : file／.ext: Need to transfer - File not found at Destination
DEBUG : sftp://user99@bnas_ts:22/usbshare1-2/zork: Waiting for checks to finish
DEBUG : sftp://user99@bnas_ts:22/usbshare1-2/zork: Waiting for transfers to finish
INFO  : file／.ext: Copied (new)

rclone ls d:\zork 
        0 file／.ext

rclone ls bnas_sftp_ts:usbshare1-2/zork 
        0 file／.ext

username78324 · November 6, 2025, 4:24pm

Try with the full directory:

> $ rclone tree test\
/
├── file／.ext
└── file：.ext

the second file should fail, not the first one.

These characters will fail differently depending on if –local-encoding None is set.

asdffdsa · November 6, 2025, 4:34pm

rclone copy d:\zork bnas_sftp_ts:usbshare1-2/zork -vv --inplace --retries=1 
DEBUG : rclone: Version "v1.71.2" starting with parameters ["d:\\data\\rclone\\rclone.exe" "copy" "d:\\zork" "bnas_sftp_ts:usbshare1-2/zork" "-vv" "--inplace" "--retries=1"]
DEBUG : Creating backend with remote "d:\\zork"
DEBUG : Using config file from "d:\\data\\rclone\\rclone.conf"
DEBUG : fs cache: renaming cache item "d:\\zork" to be canonical "//?/d:/zork"
DEBUG : Creating backend with remote "bnas_sftp_ts:usbshare1-2/zork"
DEBUG : sftp://user99@bnas_ts:22/usbshare1-2/zork: New connection 100.109.123.23:4698->100.115.37.34:22 to "SSH-2.0-OpenSSH_8.2"
DEBUG : sftp://user99@bnas_ts:22/usbshare1-2/zork: Shell type "unix" from config
DEBUG : sftp://user99@bnas_ts:22/usbshare1-2/zork: Relative path resolved to "/usbshare1-2/zork"
DEBUG : sftp://user99@bnas_ts:22/usbshare1-2/zork: Using root directory "/usbshare1-2/zork"
DEBUG : file:.ext: Need to transfer - File not found at Destination
DEBUG : file／.ext: Need to transfer - File not found at Destination
INFO  : file／.ext: Copied (new)
INFO  : file:.ext: Copied (new)

rclone tree d:\zork 
/
├── file／.ext
└── file：.ext

rclone tree bnas_sftp_ts:usbshare1-2/zork 
/
├── file／.ext
└── file：.ext

username78324 · November 6, 2025, 5:06pm

I don’t know why it’s failing for me. For testing, I took an old usb stick, formatted it to ntfs, mounted it with the following config:

UUID=94562F11562EF3A0  /mnt/usb  ntfs-3g  defaults,uid=1000,gid=1001,nofail,windows_names  0  0

I still got the same error:

> $ rclone tree test
/
└── file：.ext

0 directories, 1 files

> $ rclone sync test odroid:/mnt/usb/test --retries 1 -vv
2025/11/06 18:05:33 DEBUG : rclone: Version "v1.71.2" starting with parameters ["rclone" "sync" "test" "odroid:/mnt/usb/test" "--retries" "1" "-vv"]
2025/11/06 18:05:33 DEBUG : Creating backend with remote "test"
2025/11/06 18:05:33 DEBUG : Using config file from "C:\\Users\\fiso\\AppData\\Roaming\\rclone\\rclone.conf"
2025/11/06 18:05:33 DEBUG : fs cache: renaming cache item "test" to be canonical "//?/C:/Users/fiso/Desktop/folder/test"
2025/11/06 18:05:33 DEBUG : Creating backend with remote "odroid:/mnt/usb/test"
2025/11/06 18:05:33 DEBUG : sftp://odroid@192.168.1.25:22//mnt/usb/test: New connection 192.168.1.15:64473->192.168.1.25:22 to "SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.14"
2025/11/06 18:05:35 DEBUG : sftp://odroid@192.168.1.25:22//mnt/usb/test: Shell type "unix" from config
2025/11/06 18:05:35 DEBUG : sftp://odroid@192.168.1.25:22//mnt/usb/test: Using root directory "/mnt/usb/test"
2025/11/06 18:05:35 DEBUG : file:.ext: Need to transfer - File not found at Destination
2025/11/06 18:05:35 DEBUG : sftp://odroid@192.168.1.25:22//mnt/usb/test: Waiting for checks to finish
2025/11/06 18:05:35 DEBUG : sftp://odroid@192.168.1.25:22//mnt/usb/test: Waiting for transfers to finish
2025/11/06 18:05:35 ERROR : file:.ext: Failed to copy: Update Create failed: sftp: "Bad message" (SSH_FX_BAD_MESSAGE)
2025/11/06 18:05:35 ERROR : sftp://odroid@192.168.1.25:22//mnt/usb/test: not deleting files as there were IO errors
2025/11/06 18:05:35 ERROR : sftp://odroid@192.168.1.25:22//mnt/usb/test: not deleting directories as there were IO errors
2025/11/06 18:05:35 ERROR : Attempt 1/1 failed with 1 errors and: Update Create failed: sftp: "Bad message" (SSH_FX_BAD_MESSAGE)
2025/11/06 18:05:35 INFO  :
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Checks:                 0 / 0, -, Listed 1
Elapsed time:         0.0s

2025/11/06 18:05:35 DEBUG : 12 go routines active
2025/11/06 18:05:35 DEBUG : sftp://odroid@192.168.1.25:22//mnt/usb/test: Closing 1 unused connections
2025/11/06 18:05:35 NOTICE: Failed to sync: Update Create failed: sftp: "Bad message" (SSH_FX_BAD_MESSAGE)

> $ rclone tree odroid:/mnt/usb/test
/

0 directories, 0 files

username78324 · November 6, 2025, 5:25pm

Could you try running rclone ls d:\zork on your client machine and share the output?

For me, it prints a normal colon instead of ：, even when just listing and no remote is involved at all.

asdffdsa · November 6, 2025, 6:20pm

yeah, this output is confusing...

rclone tree d:\zork 
/
├── file／.ext
└── file：.ext

rclone ls d:\zork  
        0 file／.ext
        0 file:.ext

rclone ls d:\zork --local-encoding=None 
        0 file‛／.ext
        0 file：.ext

rclone ls d:\zork --local-encoding=Raw 
        0 file‛／.ext
        0 file：.ext

username78324 · November 6, 2025, 6:25pm

I suppose on my machine, the filenames shown by rclone ls are the same ones that are being synced, while on your machine the ls output is also weird, but the sync is somehow handled correctly. I will create a new suspected bug thread.

asdffdsa · November 6, 2025, 6:31pm

github.com/rclone/rclone

Incorrent filename encoding when uploaded from Windows local to sftp (Linux) backend

opened 01:48AM - 28 Jun 24 UTC

sagan

bug duplicate encoding

#### What is the problem you are having with rclone? Rclone incorrectly encod…es (escapes) the filename, when transferring files from local (on Windows) to sftp (on Linux) backend, even with `--local-encoding None` flag set. See below "Reproduce" section for details. #### What is your rclone version (output from `rclone version`) I have tried latest stable (rclone v1.67.0) and beta (rclone v1.68.0-beta.8049.754e53dbc), the problem can be reproduced in both. #### Which OS you are using and how many bits (e.g. Windows 7, 64 bit) Windows 10 & Windows 11, amd64 #### Which cloud storage system are you using? (e.g. Google Drive) sftp (Linux openssh) #### Reproduce I have files in Windows which filenames contain `／` (Fullwidth Solidus), `？` (FULLWIDTH QUESTION MARK) and other full width version of special characters that are restricted on Windows filenames. I want to upload these files to remote while **keeping the file name unchanged**, but with sftp backend, it's impossible. Let's say we have a directory with only one `／？.txt` ( a Fullwidth Solidus + a FULLWIDTH QUESTION MARK) file in Windows. I have a sftp backend, it's a standard Linux server with OpenSSH, rclone.conf config: ``` [testsftp] type = sftp host = 192.168.1.10 user = root shell_type = unix md5sum_command = md5sum sha1sum_command = sha1sum encoding = None pass = password ``` If I upload the file without `--local-encoding None` option: ``` rclone copy . testsftp:/tmp ``` Then, the uploaded file in Linux will have the name `／?.txt` . The original fullwidth `？` character in filename was replaced by normal question mark `?` If I upload the file with `--local-encoding None` option: ``` rclone copy --local-encoding None . testsftp:/tmp ``` Then, the uploaded file in Linux will have the name `‛／？.txt` . The `／` character was escaped to `‛／` . In both cases, the uploaded file will have it's original filename changed. #### Other info I also tried to add `encoding = None` to sftp backend config in rclone.conf but with no luck. I think this problem is specific to Windows + sftp backend. I tried upload files from Windows -> google drive, with `--local-encoding None` flag set, rclone correctly uploads file with filenames unchanged. Rclone also works when upload files from Linux -> sftp (Linux).

username78324 · November 6, 2025, 6:34pm

Oh, missed that, thanks.

username78324 · November 6, 2025, 9:12pm

I found the solution in the linked thread: I had to add --local-encoding Slash, or preferably set the environment variable RCLONE_LOCAL_ENCODING=Slash.

Doing that fixed both the escaping issue and failed transfer issue.

system · November 9, 2025, 9:12pm

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.