Reggression in v1.63.0: `--links` drops the `.rclonelink` extension

Suspected Bug
1

What is the problem you are having with rclone?

When you symlinks locally to locally with --links, you do get text files with reference to the link but they do not have .rclonelink

Run the command 'rclone version' and share the full output of the command.

rclone v1.63.0
- os/version: darwin 13.4.1 (64 bit)
- os/kernel: 22.5.0 (arm64)
- os/type: darwin
- os/arch: arm64 (ARMv8 compatible)
- go/version: go1.20.5
- go/linking: dynamic
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Local

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Setup:

$ tree src
src
├── file1.txt
└── link1.txt -> file1.txt

$ cat src/file1.txt
CONTENT

Then the rclone command:

rclone sync -vv --links src/ dst/

(see output below)

Now when you do

$ tree dst 
dst
├── file1.txt
└── link1.txt
$ cat dst/link1.txt
file1.txt

So dst/link1.txt is a text file pointing to file1.txt but is missing the .rclonelink extension.

Side note: --copy-links and --skip-links work as expected

The rclone config contents with secrets removed.

N/A -- Local to Local

A log from the command with the -vv flag

2023/06/30 15:47:36 DEBUG : rclone: Version "v1.63.0" starting with parameters ["rclone" "sync" "-vv" "--links" "src/" "dst/"]
2023/06/30 15:47:36 DEBUG : Creating backend with remote "src/"
2023/06/30 15:47:36 DEBUG : Using config file from "/Users/jwinokur/.config/rclone/rclone.conf"
2023/06/30 15:47:36 DEBUG : local: detected overridden config - adding "{b6816}" suffix to name
2023/06/30 15:47:36 DEBUG : fs cache: renaming cache item "src/" to be canonical "local{b6816}:/Users/jwinokur/rclone_demo/src"
2023/06/30 15:47:36 DEBUG : Creating backend with remote "dst/"
2023/06/30 15:47:36 DEBUG : local: detected overridden config - adding "{b6816}" suffix to name
2023/06/30 15:47:36 DEBUG : fs cache: renaming cache item "dst/" to be canonical "local{b6816}:/Users/jwinokur/rclone_demo/dst"
2023/06/30 15:47:36 DEBUG : file1.txt: Need to transfer - File not found at Destination
2023/06/30 15:47:36 DEBUG : link1.txt.rclonelink: Need to transfer - File not found at Destination
2023/06/30 15:47:36 DEBUG : Local file system at /Users/jwinokur/rclone_demo/dst: Waiting for checks to finish
2023/06/30 15:47:36 DEBUG : Local file system at /Users/jwinokur/rclone_demo/dst: Waiting for transfers to finish
2023/06/30 15:47:36 DEBUG : file1.txt: md5 = 9da8213efd566be4c7f5ebfa8d83af9a OK
2023/06/30 15:47:36 DEBUG : file1.txt.yulatef6.partial: renamed to: file1.txt
2023/06/30 15:47:36 INFO  : file1.txt: Copied (new)
2023/06/30 15:47:36 DEBUG : link1.txt.rclonelink: md5 = bd01856bfd2065d0d1ee20c03bd3a9af OK
2023/06/30 15:47:36 ERROR : link1.txt.rclonelink: Failed to read link size: readlink /Users/jwinokur/rclone_demo/dst/link1.txt: invalid argument
2023/06/30 15:47:36 DEBUG : link1.txt.rclonelink.pobedoy8.partial: renamed to: link1.txt.rclonelink
2023/06/30 15:47:36 INFO  : link1.txt.rclonelink: Copied (new)
2023/06/30 15:47:36 DEBUG : Waiting for deletions to finish
2023/06/30 15:47:36 INFO  :
Transferred:   	         17 B / 17 B, 100%, 0 B/s, ETA -
Transferred:            2 / 2, 100%
Elapsed time:         0.0s

2023/06/30 15:47:36 DEBUG : 7 go routines active
2

Yeah it is regression bug I think.

BTW it should not be .rclonelink but link itself as it is local filesystem. This is how it worked in rclone 1.62.

I have bisected it and this is commit which changed this behaviour:

5c594fea90429ccc35328a3c387b9d326167cfa9 is the first bad commit
commit 5c594fea90429ccc35328a3c387b9d326167cfa9
Author: Janne Hellsten <jjhellst@gmail.com>
Date:   Tue Apr 4 18:19:16 2023 +0300

    operations: implement uploads to temp name with --inplace to disable

    When copying to a backend which has the PartialUploads feature flag
    set and can Move files the file is copied into a temporary name first.
    Once the copy is complete, the file is renamed to the real
    destination.

    This prevents other processes from seeing partially downloaded copies
    of files being downloaded and prevents overwriting the old file until
    the new one is complete.

    This also adds --inplace flag that can be used to disable the partial
    file copy/rename feature.

    See #3770

    Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>

 docs/content/docs.md                 | 43 ++++++++++++++++++++++++++
 fs/config.go                         |  1 +
 fs/config/configflags/configflags.go |  1 +
 fs/operations/operations.go          | 55 ++++++++++++++++++++++++++++++----
 fs/operations/operations_test.go     | 58 ++++++++++++++++++++++++++++++++++++
 5 files changed, 152 insertions(+), 6 deletions(-)
3

You can have it working like in 1.62 by adding --inplace flag.

I am not entirely sure if this is intended behaviour of 1.63 or bug.

But I lean towards bug:) with or without --inplace final result should be always the same.

1 Like
4

Sounds like a bug to me. Will investigate.

5

I think this is the same issue: reggression in v1.63.0: .rclonelink file are not converted back to symlinks, even with `--links` · Issue #7101 · rclone/rclone · GitHub

1 Like
6

Indeed. Slightly different use case but the same root cause.

7

Please try this fix

v1.64.0-beta.7121.8ccf016af.fix-7101-partial-link on branch fix-7101-partial-link (uploaded in 15-30 mins)

8

Using the same use case as in the original post I got:

$ tree src/
src/
├── link1 -> text1.txt
└── text1.txt

rclone sync --links src/ dst/

$ tree dst/
dst/
├── link1 -> text1.txt
└── text1.txt

vs 1.63

$ tree dst/
dst/
├── file1.txt
└── link1.txt

so now it works like in v1.62

9

Thanks for testing :slight_smile: I'll put this into a v1.63.1 quite soon!

2 Likes
10

Thanks! Sorry I couldn’t test. Family full weekend

11

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.