```rclone move chunker:dir/file chunker:dir``` deletes all file chunks

Suspected Bug
1

What is the problem you are having with rclone?

Moving a chunked-file to its own directory deletes all of the file's chunks.

rclone version

rclone v1.65.0
- os/version: debian 12.1
- os/kernel: 6.1.21+ (armv6l)
- os/type: linux
- os/arch: arm (ARMv6 compatible)
- go/version: go1.21.4
- go/linking: static
- go/tags: none

Used remotes:

chunker over local file system

Command

rclone move chunker:dir/file chunker:dir

Config

[chunker]
type = chunker
remote = /home/MyUsername
chunk_size = 1Mi

Steps to reproduce:

1. Have some chunked file inside chunker remote.

$ rclone tree chunker:

/
└── rclone.deb

0 directories, 1 files

2. Move file to its own directory using the command below.

$ rclone move chunker:rclone.deb chunker: -vv

<7>DEBUG : rclone: Version "v1.65.0" starting with parameters ["rclone" "move" "chunker:rclone.deb" "chunker:" "-vv"]
<7>DEBUG : rclone: systemd logging support activated
<7>DEBUG : Creating backend with remote "chunker:rclone.deb"
<7>DEBUG : Using config file from "/home/MyUsername/.config/rclone/rclone.conf"
<7>DEBUG : Creating backend with remote "/home/MyUsername/chunker/rclone.deb"
<7>DEBUG : fs cache: adding new entry for parent of "/home/MyUsername/chunker/rclone.deb", "/home/txtr/chunker"
<7>DEBUG : Reset feature "ListR"
<7>DEBUG : Creating backend with remote "chunker:"
<7>DEBUG : Reset feature "ListR"
<7>DEBUG : rclone.deb: Size and modification time the same (differ by 0s, within tolerance 1ns)
<7>DEBUG : rclone.deb: Unchanged skipping
<6>INFO  : rclone.deb: Deleted
<6>INFO  : 
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:                 1 / 1, 100%
Deleted:                1 (files), 0 (dirs)
Elapsed time:         0.0s

<7>DEBUG : 4 go routines active

Result:

$ rclone tree chunker:

/

0 directories, 0 files

Expected result:

$ rclone tree chunker:

/
└── rclone.deb

0 directories, 1 files
1 Like
2

Indeed I can replicate it easily.

I think it is a bug. Not massive but still one.

3

i would expect rclone to output.

don't need to copy/move rclone.deb, it is already at target location

4

E.g. for local "remote" nothing happens:

$ rclone lsf .
a.txt

$ rclone move ./a.txt .

$ rclone lsf .
a.txt

but for chunker remote result is file deleted without any error.

So definitely rclone behaviour is not consistent .

5

need to post a debug log

rclone move wasabi01:zork/file.ext wasabi01:zork -vv
2023/12/07 13:29:22 DEBUG : rclone: Version "v1.65.0" starting with parameters ["rclone" "move" "wasabi01:zork/file.ext" "wasabi01:zork" "-vv"]
2023/12/07 13:29:22 DEBUG : Creating backend with remote "wasabi01:zork/file.ext"
2023/12/07 13:29:22 DEBUG : Using config file from "c:\\data\\rclone\\rclone.conf"
2023/12/07 13:29:22 DEBUG : Resolving service "s3" region "us-east-2"
2023/12/07 13:29:22 DEBUG : fs cache: adding new entry for parent of "wasabi01:zork/file.ext", "wasabi01:zork"
2023/12/07 13:29:22 DEBUG : S3 bucket zork: don't need to copy/move file.ext, it is already at target location
1 Like
6

Yes I agree, this is a bug.

I investigated and it turns out its a bug in the chunker backend.

It was easy to fix though - please give this a try

v1.66.0-beta.7565.88a3de4a7.fix-chunker-file on branch fix-chunker-file (uploaded in 15-30 mins)

1 Like
7

FYI - looks like these builds are missing osx-amd-64 recently

8

the beta seems to have fixed the issue

./rclone touch chunker:file.ext

./rclone move chunker:file.ext chunker: -vv
2023/12/08 12:57:59 DEBUG : rclone: Version "v1.66.0-beta.7565.88a3de4a7.fix-chunker-file" starting with parameters ["./rclone" "move" "chunker:file.ext" "chunker:" "-vv"]
2023/12/08 12:57:59 DEBUG : Creating backend with remote "chunker:file.ext"
2023/12/08 12:57:59 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/12/08 12:57:59 DEBUG : Creating backend with remote "/root/test/file.ext"
2023/12/08 12:57:59 DEBUG : fs cache: adding new entry for parent of "/root/test/file.ext", "/root/test"
2023/12/08 12:57:59 DEBUG : Reset feature "ListR"
2023/12/08 12:57:59 DEBUG : fs cache: adding new entry for parent of "chunker:file.ext", "chunker:"
2023/12/08 12:57:59 DEBUG : Chunked 'chunker:': don't need to copy/move file.ext, it is already at target location
2023/12/08 12:57:59 INFO  :
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Elapsed time:         0.0s

2023/12/08 12:57:59 DEBUG : 5 go routines active
9

Works for me as well - with chunker on top of crypt and union. It did not work before.

10

Thanks for testing :slight_smile:

I made an integration test for this and it showed the compress and crypt backends are broken in the same way, so I'll have to fix those too.

1 Like
11

The fix for this is merged into the latest beta
and I'll put it in 1.65.1

1 Like
12

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.