Some unicode forms break mount on macOS with FUSE-T

[jwink3101 (Justin Winokur)]

What is the problem you are having with rclone?

Certain unicode normalized files are breaking my mount. I thought this was an issue with FUSE-T and created Cannot handle certain normalized forms of unicode. Example: é (e + U+0301) vs é (U+00E9) · Issue #16 · macos-fuse-t/fuse-t · GitHub but I am less certain now.

Example:

• breaks: é . That is e + U+0301 or UTF8 encoded: e\xcc\x81
• works: é which is (U+00E9) or \xc3\xa9 in UTF8

When they are in the directory, I get

$ ls
ls: fts_read: Permission denied

and nothing shows in Finder. As seen in the logs below and the one in the aforementioned issue, rclone appears to read it fine which is why I thought this was a FUSE-T issue. But both the developer's and my own additional FUSE-T testing shows that is less likely the case.

Looking at the log, I see Adding "-o modules=iconv,from_code=UTF-8,to_code=UTF-8-MAC" for macOS. Is there a way to disable this and see if that i the problem?

Run the command 'rclone version' and share the full output of the command.

rclone v1.61.1
- os/version: darwin 13.2.1 (64 bit)
- os/kernel: 22.3.0 (arm64)
- os/type: darwin
- os/arch: arm64
- go/version: go1.19.4
- go/linking: dynamic
- go/tags: cmount

I am also using the latest FUSE-T according to brew

Which cloud storage system are you using? (eg Google Drive)

Originally discovered on a crypt-wrapped local but reproduced on a simple local loopback mount.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount source/ mount/ -vv --log-file rclone.log

The rclone config contents with secrets removed.

N/A -- Local loopback. 

A log from the command with the -vv flag

rclone.log (29.2 KB)

[ncw (Nick Craig-Wood)]

Does this break with OSXFUSE too?

@jwink3101

Looking at the log, I see Adding "-o modules=iconv,from_code=UTF-8,to_code=UTF-8-MAC" for macOS. Is there a way to disable this and see if that i the problem?

What that does is map the unicode that rclone (UTF-8) uses to the decomposed unicode that macOS uses internally (UTF-8-MAC). Its a command to libfuse.

If you want to disable this then use -o modules=iconv,from_code=UTF-8,to_code=UTF-8 - I think that should work.

I wonder if FUSE-T doesn't use decomposed unicode internally?

macOS decomposed unicode (NFD) is a PITA - you'll find dozens of issues relating to this on the rclone Github!

[jwink3101 (Justin Winokur)]

@ncw

Does this break with OSXFUSE too?

I don't have a machine that has the older one. I can see if a virtual works when I am home later. But both my work mac and my home one uses FUSE-T now.

@ncw

If you want to disable this then use -o modules=iconv,from_code=UTF-8,to_code=UTF-8 - I think that should work.

That does work! I first confirmed that my work mac does this too (it's Intel and I have ARM at home so I wanted to be sure) and confirmed that this flags fixes it. I will add it to my mount script (though I already just renamed the file since I don't care about the "é".

I'll update the issue on Github for FUSE-T as well