Characters in filenames are off when using Synology via SMB

Help and Support
1

What is the problem you are having with rclone?

Some characters, like the ? character, are causing problems when trying to use rclone between my Mac and my Synology over SMB. The characters appear normal in Finder when I connect directly via SMB, but when running rclone the ? character on the synology gets converted to a ? inside a box (not sure what the actual name for that character is). The character also appears correctly when I log into the Synology via its web UI. For further context, I had previously copied these files with another tool and am attempting to retrofit rclone onto this archive as a replacement.

Run the command 'rclone version' and share the full output of the command.

rclone v1.66.0
- os/version: darwin 14.5 (64 bit)
- os/kernel: 23.5.0 (arm64)
- os/type: darwin
- os/arch: arm64 (ARMv8 compatible)
- go/version: go1.22.1
- go/linking: dynamic
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Self-hosted Synology, connecting via SMB

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone check Podcasts/snippets synology:home/T7/Podcasts/snippets --local-encoding None --smb-encoding None

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[synology]
type = smb
host = XXX
user = XXX
pass = XXX

A log from the command that you were trying to run with the -vv flag

2024/05/16 18:20:56 DEBUG : rclone: Version "v1.66.0" starting with parameters ["rclone" "check" "Podcasts/snippets" "synology:home/T7/Podcasts/snippets" "-vv" "--log-file=log.txt"]
2024/05/16 18:20:56 DEBUG : Creating backend with remote "Podcasts/snippets"
2024/05/16 18:20:56 DEBUG : Using config file from "/Users/user/.config/rclone/rclone.conf"
2024/05/16 18:20:56 DEBUG : fs cache: renaming cache item "Podcasts/snippets" to be canonical "/Volumes/T7/Podcasts/snippets"
2024/05/16 18:20:56 DEBUG : Creating backend with remote "synology:home/T7/Podcasts/snippets"
2024/05/16 18:20:57 DEBUG : smb://user@ds218.local:445/home/T7/Podcasts/snippets: Using root directory "home/T7/Podcasts/snippets"
2024/05/16 18:20:57 ERROR : No common hash found - not using a hash for checks
2024/05/16 18:20:57 DEBUG : smb://user@ds218.local:445/home/T7/Podcasts/snippets: Waiting for checks to finish
2024/05/16 18:20:57 ERROR : 2024-02-23 - Where did all the roaches go?.mp3: file not in smb://user@ds218.local:445/home/T7/Podcasts/snippets
2024/05/16 18:20:57 ERROR : 2024-02-23 - Where did all the roaches go.mp3: file not in Local file system at /Volumes/T7/Podcasts/snippets
2024/05/16 18:20:57 NOTICE: smb://user@ds218.local:445/home/T7/Podcasts/snippets: 1 files missing
2024/05/16 18:20:57 NOTICE: Local file system at /Volumes/T7/Podcasts/snippets: 1 files missing
2024/05/16 18:20:57 NOTICE: smb://user@ds218.local:445/home/T7/Podcasts/snippets: 2 differences found
2024/05/16 18:20:57 NOTICE: smb://user@ds218.local:445/home/T7/Podcasts/snippets: 2 errors while checking
2024/05/16 18:20:57 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Errors:                 2 (retrying may help)
Elapsed time:         0.3s

2024/05/16 18:20:57 DEBUG : 8 go routines active
2024/05/16 18:20:57 DEBUG : smb://user@ds218.local:445/home/T7/Podcasts/snippets: Closing 1 unused connections
2024/05/16 18:20:57 Failed to check with 2 errors: last error was: 2 differences found
2

Most likely culprit is character encoding:

You can configure it to your specific needs but unfortunately there is no one setting which works for all. By default rclone tries to be as compatible as possible for cross platform environments.

Good example is ？ and ?. Former works on all systems but latter wont work on Windows (where ? is restricted character).

3

I think I've found that changing encodings works on rclone sync, but not on rclone check which was part of my confusion. But even that isn't sufficient for sync (and I'm unsure if it's a bug or intended).

If I sync a new file with proper encoding flags, then ? will remain unchanged in the file name. However, if run the exact same command again, sync will re-copy those files because it's technically not the same on check, but then it will delete them. Here's what happens:

Synology:

file1.txt

Local:

file1.txt
file2?.txt

Then, run rclone sync with encodings set to None. This works as expected:

Synology and local are both:

file1.txt
file2?.txt

However, this is not idempotent. If I run the same rclone sync command again, it actually re-copies and then deletes file2?.txt from the synology and I'm back where I started.

If this is intended behavior, then I will investigate alternative tools that are better suited for my use-case. But this does kind of feel like a bug. Any thoughts?

4

Characters encoding on macOS is one big bug:slight_smile: - and some more background here.

It works perfectly as long as you are using macOS only - but multiple issues surface when you share your files with different OS systems... It is not rclone limitation but consequence of Apple design decisions.

In such cases what I would suggest is to make sure that all your NAS stored files use NFD normalisation (especially if files originate from other systems). You can easily convert them using convm (it should be part of most Linux distros):

convmv -r -f utf8 -t utf8 --nfd --notest .

Try with both local and smb encoding set to None:

--smb-encoding = "None"
--local-encoding = "None"
5

I would also try to use macOS smb mount instead of rclone smb - the latter is based on golang smb implementation which might have its own quirks and bugs as it is not as thoroughly used and tested as macOS SMB version.

Then maybe try NFS or Webdav instead of SMB?

I do not have Synology NAS now so can't test myself what is the best approach here.

6

Thanks for the info about macOS character encoding. There were some other issues with using the builtin SMB mount and with SFTP.

And sorry for not clarifying, but those commands had both smb and local encodings set to None.

Considering some other tools I've used that handled syncing between macOS and this synology fine, my gut feeling is that something with the go library may be causing issues, or maybe just the fact that rclone is designed for cloud storage first and foremost, so I think it's just not the right tool for my use case.

Thanks for helping me understand rclone better!

1 Like
7

welcome to the forum,

fwiw, i do not use rclone smb remote, instead i always use native os tools.
not sure it will make a difference in your case.

check out my howto guide about rclone and smb.
https://forum.rclone.org/t/how-to-access-smb-samba-with-rclone/42754

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.