Rclone recognizing certain filenames incorrectly

Suspected Bug
1

There seems to be a bug that is causing the sync to redownload the same file over and over. I'm trying to make a backup of my Google Drive to my Local NAS (SMB protocol).

For example when I sync:
rclone sync google:Library/Movies/Movies nas:data/media/movies/movies-Archive --progress

99% of the files are syncing just fine. But some files keeps resyncing every time when i start sync again.

I debugged a little bit by running a LS command on the affected movie.

When i run:
rclone ls google:"Library/Movies/Movies/Movie (2018)"

I get:
117964 Movie.srt
8211670794 Movie.mkv

But when running:
rclone ls nas:"data/media/movies/movies-Archive/Movie (2018)"

I get this:
8211670794 C38RC3~3.MKV
117964 CPJOWA~Y.SRT

The files have all a wrong name. But the weird thing is on the NAS are the filenames correct. I don't know what is causing this behavior.

2

welcome to the forum,

let's figure that out.
copy a single file using rclone copy --dry-run -vv and post the full output.

3

As expected:

2024/05/11 18:02:07 DEBUG : rclone: Version "v1.66.0" starting with parameters ["rclone" "copy" "google:Library/Movies/Movies/Movie (2018)" "nas:data/media/movies/movies-Archive/Movie (2018)" "--dry-run" "-vv"]
2024/05/11 18:02:07 DEBUG : Creating backend with remote "google:Library/Movies/Movies/Movie (2018)"
2024/05/11 18:02:07 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/05/11 18:02:07 DEBUG : google: Loaded invalid token from config file - ignoring
2024/05/11 18:02:08 DEBUG : Saving config "token" in section "google" of the config file
2024/05/11 18:02:08 DEBUG : Keeping previous permissions for config file: -rw-r--r--
2024/05/11 18:02:08 DEBUG : google: Saved new token in config file
2024/05/11 18:02:09 DEBUG : Creating backend with remote "nas:data/media/movies/movies-Archive/Movie (2018)"
2024/05/11 18:02:09 DEBUG : smb://xxx@10.0.0.2:445/data/media/movies/movies-Archive/Movie (2018): Using root directory "data/media/movies/movies-Archive/Movie (2018)"
2024/05/11 18:02:10 DEBUG : Movie.srt: Need to transfer - File not found at Destination
2024/05/11 18:02:10 DEBUG : Movie.mkv: Need to transfer - File not found at Destination
2024/05/11 18:02:10 NOTICE: Movie.srt: Skipped copy as --dry-run is set (size 115.199Ki)
2024/05/11 18:02:10 NOTICE: Movie.mkv: Skipped copy as --dry-run is set (size 7.648Gi)
2024/05/11 18:02:10 DEBUG : smb://xxx@10.0.0.2:445/data/media/movies/movies-Archive/Movie (2018): Waiting for checks to finish
2024/05/11 18:02:10 DEBUG : smb://xxx@10.0.0.2:445/data/media/movies/movies-Archive/Movie (2018): Waiting for transfers to finish

LS on the NAS itself returns the correct results.

4

to keep things simple, let's focus on one single file, Movie.srt

afiak, based on the output of rclone ls logs, rclone is performing the correct action.
the source file does not exist in the dest, so rclone wants to transfer it.

5

But the problem is that both files exists on the NAS. but due to a unknown reason rclone cannot read the files properly on the NAS.

I created a test example to proof the bug:

I created this TXT-file on the NAS:
Con.Man.2018.STV.1080p.BluRay.x264-TheWretched.en.txt

But Rclone seems to see this:

6 CPJOWA~Y.TXT

I't very weird because when I create:
Movie.BluRay.x264-TheWretched.en.txt

Then Rclone sees it correctly:

4 Movie.BluRay.x264-TheWretched.en.txt

I't seems to be related to filenames.

6

ok, now, i see the issue.

that is old-school 8.3 dos format.

can you post the output of rclone config redacted ?

7

But it has to be SMB related.

[google]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[nas]
type = smb
host = XXX
user = XXX
pass = XXX
### Double check the config for sensitive info before posting publicl
8

The rclone client tries to avoid 8.3 names when uploading files by encoding trailing spaces and periods. Alternatively, the local backend on Windows can access SMB servers using UNC paths, by \\server\share

what OS is that nas running?

fwiw, i do not use rclone smb remote, as it lacks many basic features, including file verification using checksums.
my backup script always used native OS tools, on both windows and linux.

i have a how-to guide about all that, which includes sample scripts.
https://forum.rclone.org/t/how-to-access-smb-samba-with-rclone/42754

9 
mount -t cifs -o username="xxx",password="xxx" //10.0.0.2/data /mnt/smb

I mounted my SMB share local. (As the 'how to' describes) But the problem still exists.

I'm running latest Unraid 6.12. I don't know what SMB-version they are using.

rclone ls "/mnt/smb/media/movies/movies-Archive/Movie (2018)"
8211670794 C38RC3~3.MKV
   117964 CPJOWA~Y.SRT
        4 Movie.BluRay.x264-TheWretched.en.txt
10

ok, nice, we are getting closer and now, can reproduce on a single machine.

afaik, some sort of name-mangling is going on, mabye samba, maybe unraid, maybe ...
my next step might be to check smb.conf, maybe there is a hint

11

As for Samba. My Unraid-NAS is using 'Samba version 4.17.12'

Extra info about active connections:

659   xxx       users        10.0.0.4 (ipv4:10.0.0.4:36570)            SMB3_11           -                    partial(AES-128-CMAC)

Running the LS command on a SMB mounted share is also incorrect.

This topic can be closed because it's not rclone related.

I did found a solution:
https://www.reddit.com/r/unRAID/comments/1bwmnxh/smb_folder_name_issues/

12

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.