Rclone is trying to sync symbolic links even when it has been added to exclusion

What is the problem you are having with rclone?

I'm synching an entire hard drive between local and GDrive with links (Symlinks) included. This drive has a symlink which gives an error in Windows that says "The file or directory is corrupted or unreadable." so I can't make any read operations with it thus I'm excluding it from the sync operation.

Now, I have added an entry to the exclusion command -and this entry is the aforementioned symbolic link- but rclone is ignoring this and is still trying to sync that "file". Other exclusions work as usual, it seems rclone is ignoring exclusions that point to symlinks when we use --links.

What is your rclone version (output from rclone version)


Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync E:\ "crypt:Drive (E:)" --delete-after --checksum --no-update-modtime --transfers 4 --checkers 20 --contimeout 60s --timeout 300s --retries 3 --low-level-retries 10 --delete-excluded --exclude "/dir1/subdir1/subdir2/symlink.ext" --create-empty-src-dirs --backup-dir crypt:Removed --links --track-renames --track-renames-strategy modtime --drive-chunk-size 256M --fast-list

A log from the command with the -vv flag

debug.log (112.0 KB)

Thais is the last I got before rclone exited (It made 3 attempts to read the symlink).

You didn't include a log file so we have no idea what error you are seeing :frowning:

Sorry, I have added it.

That's a snippet of a log so there's no command and it's not a debug log.

Can please include the full debug log?

Aaah, rigth, I used --verbose instead, whoops. I added the debug log as a file because it's big.

If you want to make it easier, just pick one folder or where you see the problem and test that out to make it easier to look at as well.

That's what I did because if I shared the debug log with the original local path it would have been much larger. I changed the path to the parent dir of the directory where the file is located

The error says you can't read the location which it has to read it to exclude it.

El archivo o directorio está dañado o es ilegible.


The file or directory is corrupted or unreadable.

You can't skip/exclude something if you can't read into the directory so you aren't making it to your exclude.

But does it really needs to read the file? I mean, I don't know the code obviously so maybe I'm saying something pretty stupid but if we locate in Windows, to know the contents of a directory Windows doesn't need to read all the files that said directory contains, running dir I can get the file listing without getting any error because -as far as I understand- it doesn't read any file.

Even so, I have other two files that doesn't have the same but a CRC error (drive has faulty sectors) and I have excluded them with success, why I'm not getting the same results (an error trying to read them) with those files if rclone needs to read them before excluding them?

It isn’t trying to read the file as it’s trying to navigate into the path and see what files exist.

You can’t exclude something you don’t have access to. By read, I mean read the directory and file metadata and not actually reading the file.

Doesn't dir reads the metadata? Even file explorers read metadata and I can view the symlink metadata just fine. This is what it shows when I run the command:

18/11/2021  17:16    <SYMLINK>      FILENAME.EXT [..]

But also, why does it needs to read the metadata before excluding it? With --exclude the only thing it needs to know is the file path/name, as long as that matches the exclude param then it just needs to ignore it no matter what metadata it contains, isn't it?

Thanks for helping me

I feel like we're spinning in circles here.

Rclone needs permissions / access to walk a directory path to exclude things. If it can't, you can't exclude it and it will generate your error.

Fix the root cause here or live with the error.

Yeah, it seems so.
As I understand you first told me it needs to read the metadata of every item. It can read the metadata of the symbolic link because other software can.
Then you told me it needs access to the directory path to view its content, it can access the path and its content can be listed because other software can.

You can’t exclude something you don’t have access to. By read, I mean read the directory and file metadata and not actually reading the file.

Rclone should be able to access the directory and to read the metadata of the symlink, why wouldn't it be able to if other software can do it?
What no software can do is resolve the link nor can the symlink be deleted/renamed/opened, none of which rclone should be trying to do and for what I read in the log it is trying to read the symlink I'm guessing to resolve it.

The problem that I've reported is that rclone is not excluding symlinks no matter if rclone can or not resolve the link.

If I tree the directory using rclone I get:

rclone tree --human --color --modtime --links "E:\dir1\subdir1\subdir2"

2021/11/26 22:23:56 ERROR : FILENAME.ext.rclonelink: Failed to read link size: readlink \\?\E:\dir1\subdir1\subdir2\FILENAME.ext: El archivo o directorio está dañado o es ilegible.

├── [          Nov 18 17:16]  FILENAME.ext.rclonelink

But it also tries to read the link and I don't get why when all that it's showing are the dates and the filenames, it should be enough for rclone to know that it is a symbolic link and whatever it is pointing to has nothing to do with the listing. I'm misunderstanding something here?

Ahhhhh I think I got why it is failing. It seems rclone is trying to resolve the links to store the target in a text file and get the rclonelink file, that's why when I add --size to the tree command it reports sizes for all the other symbolic links except the one that is corrupted that says 0

I just go by the data. The error below says it can't read or is corrupted.

El archivo o directorio está dañado o es ilegible.

The file or directory is corrupted or unreadable.

Right as you told it to via --links:

  -l, --links                                        Translate symlinks to/from regular files with a '.rclonelink' extension

So to do that, it has to traverse it and if it gets access denied, it can't filter.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.