Issues with special characters in filenames

Help and Support
1

What is the problem you are having with rclone?

I am trying to copy a large number of music files to a NAS. The problem is that many of the files have special characters, e.g. å, and these cannot be changed without screwing up a whole series of things (playlists, interconnections, etc.). How can I get these files to copy with rclone?

What is your rclone version (output from rclone version)

1.52.0

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Catalina, 64-bit

Which cloud storage system are you using? (eg Google Drive)

Synology NAS

The command you were trying to run (eg rclone copy /tmp remote:tmp)

The rclone config contents with secrets removed.

Paste config here

A log from the command with the -vv flag

Paste  log here
2

There isn't much to do if the source and destination don't like the ones you are using.

You can take a look at:

https://rclone.org/local/#filenames

If you run a few test files that you think cause issues, are you seeing any problems?

Run a few tests with -vv and you should be able to examine the output and see what it is replacing or not replacing.

Generally, I avoid any of those characters since it tends to be problematic.

3

It's not the source or destination since I can drag and drop those files from the source to the destination with no problem. None of the file names with these special characters, e.g. ç, are being replaced at all, I just get an error code saying no such file or directory exists, but I can clearly see that the files do exist. Only files with those kinds of special characters are throwing out errors.

4

If you can post the command you are using a debug log, that would be superb.

5

Command: rclone copy -vP --stats=10s /Volumes/usbshare1-2/Music/ /Volumes/Public/Music

Error example (note special characters in Jose and Gonzalez): 2020-06-06 13:15:14 ERROR : iTunes/New Library files: lstat /Volumes/usbshare1-2/Music/iTunes/New Library files/José González: no such file or directory

Not sure how to get a log

6

You need to run the command with -vv so we can get a debug log.

You can use --log-file somefile.log

Rather than do a whole sync, just copy on directory/file you know has a problem to simplify the log and the amount of data in the log.

Are you using any Cloud remotes as well or are you just copying from a local disk to a mounted NAS disk on your Mac?

7

I wasn't sure how to just transfer one file (I tried putting a path to the file with the name in quotations but it didn't seem to recognize it), so I'm running the whole thing now with those debug log commands.

I'm not using any Cloud remotes right now, I'm just trying to copy from a local disk to a mounted NAS.

8

For local copying, there are better tools than rclone. I only rclone when I need to deal with remotes or cloud storage.

For moving one file, you'd want to change into the directory and perhaps copy that one 'bad' directory that is causing you problems. The whole log is fine to look at as well.

9

Animosity,

What tools would you recommend for local copy?

I am planning to use rclone to backup my entire NAS on google drive. However, I imagine I'll run into the same issue with special characters, eh?

Waiting on the log now.

EDIT: quesiton, am I supposed to run both -vv and the -log-file somefile.log commands? Do they need to be in any particular order? I'm trying it now, but when I add those in nothing is happening. This is what I typed:

rclone copy -vP --stats=10s --ignore-existing -vv --log-file somefile.log /Volumes/usbshare1-2/Music/ /Volumes/Public/Music/

10

The -v could be removed, but the -vv should take precedence.

For local on Linux/Mac, rsync is a nice fit.

This one " Copy/Sync a Directory on Local Computer" matches you.

https://www.tecmint.com/rsync-local-remote-file-synchronization-commands/

11

Thank you, I'm trying rsync now.

I still want to use rclone to backup to Google Drive. Do you expect I'll run into the same issues with special characters? If so, can I use rsync instead (if so, what's the advantage of rclone)?

12

So my take is use the best tool for the job.

If I'm doing local file system stuff, rsync for me is a good fit.
If I'm doing actual backups on my laptop, I like using Arq to connect to my Google Drive.

If I want to run encrypted content on my Google Drive and have that content accessible on a my systems, rclone is a superb tool for that. Generally, if you have anything you want to do with a cloud remote, rclone usually is the swiss army knife for those tasks.

I'd like to see what the error is in the debug log and we can probably figure out what the cause is. I'm not an expert by any means on unicode and how that stuff gets translated but I'm sure I can point us the right way and other folks can chime in as well.

For me, pick the best tool for the job and that choice is very personal to folks so what may work best for me, may not be best for you so I just share my experiences and you can pick :slight_smile:

1 Like
13

Animosity,

Thank you, I appreciate it. So now I'm running rsync to backup to my NAS and it seems to be working.

Once that is done, I'll try to use rclone to sync to Google Drive. If you don't mind, since you're here, I have a couple of additional questions. First, if I want the files on Google Drive to be encrypted, do I need to enter an additional command? Second, I've read about upload limits to GDrive, but there's mixed info about it. If I try to upload my entire NAS (about 4GB at the moment), will I run into problems?

EDIT: actually, it looks like something similar is happening with rsync, expect now it's saying that the files with special characters have "vanished." Ugh.

14

https://rclone.org/crypt/

that would be a question for synology forum.

15

It's not the synology though since I can drag and drop those files just fine. It's only when using rclone or rsync that I'm running into issues with the special characters. Unless I'm missing something, which is the most likely answer.

16

sorry, what i meant was
"the files with special characters have "vanished" was rsync not rclone.
https://www.synology.com/en-global/knowledgebase/DSM/help/DSM/AdminCenter/file_rsync

https://rclone.org/commands/rclone_copyto/

and add -vv to the command for debug output.

17

Restic is a nice tool for backups as well and uses rclone as a backend. That being said, I'm not sure the answer as to the special characters but it's easy enough to test out.

18

@gdthecomp I use special characters in Windows all the time, and they show up just fine in GDrive. Never had any issues with that. I encrypt everything.

19

Is there some reason to use Restic instead of rclone?

20

Yes. It's an actual backup tool. Rclone is a syncing tool. Restic does things like incremental backup sets and manages those backups. Deduplication. Verify backups. Etc.