Using `rclone moveto` to rename a file inside chunker over union of B2 buckets doesn't rename chunks (just renames metadata file)

The problem :

Using rclone moveto to rename chunked file over union of Backblaze B2 buckets doesn't rename chunks (just renames metadata file).

rclone version :

rclone v1.65.0

  • os/version: ubuntu 22.04 (64 bit)
  • os/kernel: 6.1.58+ (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.21.4
  • go/linking: static
  • go/tags: none

Cloud storage system :

                       /---> b2_account_1:bucket_1
                      /----> b2_account_2:bucket_2
                     /-----> b2_account_3:bucket_3
chunker: ----> union ------> b2_account_4:bucket_4
                     \-----> b2_account_5:bucket_5
                      \----> b2_account_6:bucket_6
                       \---> ...

Command :

rclone moveto chunker:a.ext chunker:b.ext

The rclone config contents with secrets removed.

[<b2_key_id_1>]
type = b2
account = <b2_key_id_1>
key = <b2_key_secret_1>
hard_delete = true
upload_concurrency = 1

[<b2_key_id_2>]
type = b2
account = <b2_key_id_2>
key = <b2_key_secret_2>
hard_delete = true
upload_concurrency = 1

... (lots of remotes)


[union]
type = union
upstreams = <b2_key_id_1>:<b2_bucket_1> <b2_key_id_2>:<b2_bucket_2> ...
action_policy = epff
create_policy = rand
search_policy = epall
cache_time = 604800

[chunker]
type = chunker
remote = union:
chunk_size = 1Gi

A log from the command with the -vv flag

Difficult to reproduce (but whenever bug occurs no error is reported) (I might add something later)

(Also I think bug is independent of union or b2 remotes. Trying to reproduce in local...)

Observed Result :

Only metadata file is renamed i.e.

<b2_key_id_1>:<b2_bucket_1>/a.ext becomes <b2_key_id_1>:<b2_bucket_1>/b.ext

All chunks are NOT renamed:

<b2_key_id_1>:<b2_bucket_1>/a.ext.rclone_chunk.001
<b2_key_id_1>:<b2_bucket_1>/a.ext.rclone_chunk.002
<b2_key_id_1>:<b2_bucket_1>/a.ext.rclone_chunk.003
<b2_key_id_1>:<b2_bucket_1>/a.ext.rclone_chunk.004
<b2_key_id_1>:<b2_bucket_1>/a.ext.rclone_chunk.005
...

remain of the same name.

Expected Result :

All chunks should also be renamed along with the metadata file. i.e.

<b2_key_id_1>:<b2_bucket_1>/a.ext.rclone_chunk.001 -> <b2_key_id_1>:<b2_bucket_1>/b.ext.rclone_chunk.001
<b2_key_id_1>:<b2_bucket_1>/a.ext.rclone_chunk.002 -> <b2_key_id_1>:<b2_bucket_1>/b.ext.rclone_chunk.002
...

This is getting difficult to reproduce. Maybe someone help me :slight_smile:

I have submitted another bug that may be related (`rclone moveto` errors for files in "chunker over union"). But this bug might not be completely encompassed by the other one. So, I would advise against closing this one if the other is closed.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.