Include/exclude hard linked files

Issue summary

I only want to copy files whose link count = 1, i.e. exclude hard linked files.

Similarly, I also want to copy hard linked files only.

Is there a flag that allows this to happen? I've gone through through the docs, and I can't seem to find this.

rclone version

rclone v1.54.0
- os/arch: linux/arm64
- go version: go1.15.7

There's no flag in rclone to identify how many liinks a file has.

I'm not aware of any easy way to address that other than checking the count of links on every file via scripting.

What does rsync do here? That is a question I often ask when thinking about new features...

The answer is it has a -H flag to preserve hardlinks. Is that what you want to do?

Hey Nick! How've you been?

If its not specified, does it handle hardlinks the same way rclone does? If so, hard linked files are already synced?

If you sync a hard linked file then rclone will sync it, but it will break the hardlink. So if you had both a and b hardlinked together, rclone would sync those as two seperate files a and b. This is great for data integrity but not so much for data storage.

This is what rsync does too without the -H flag.

What is your use case for hard links?

That makes sense.

I do not want to sync files that are still seeding (hard-linked) but want to sync files that are not hard-linked.

I normally sync with --min-age 14d, but I end up not removing the seeding files and sometimes my storage gets low enough that I want to transfer files that aren't seeding.

You could use a bit of scripting do make an --exclude-from file, something like this

find /root/of/transfer -type f -links +1 -printf "/%P\n" > exclude-from-file
rclone sync /root/of/transfer remote: --exclude-from exclude-from-file

That should work hopefully!

1 Like

That is very helpful, thank you!

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.