The problem :
Using rclone moveto
to rename chunked file over union of Backblaze B2 buckets doesn't rename chunks (just renames metadata file).
rclone version
:
rclone v1.65.0
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 6.1.58+ (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.4
- go/linking: static
- go/tags: none
Cloud storage system :
/---> b2_account_1:bucket_1
/----> b2_account_2:bucket_2
/-----> b2_account_3:bucket_3
chunker: ----> union ------> b2_account_4:bucket_4
\-----> b2_account_5:bucket_5
\----> b2_account_6:bucket_6
\---> ...
Command :
rclone moveto chunker:a.ext chunker:b.ext
The rclone config contents with secrets removed.
[<b2_key_id_1>]
type = b2
account = <b2_key_id_1>
key = <b2_key_secret_1>
hard_delete = true
upload_concurrency = 1
[<b2_key_id_2>]
type = b2
account = <b2_key_id_2>
key = <b2_key_secret_2>
hard_delete = true
upload_concurrency = 1
... (lots of remotes)
[union]
type = union
upstreams = <b2_key_id_1>:<b2_bucket_1> <b2_key_id_2>:<b2_bucket_2> ...
action_policy = epff
create_policy = rand
search_policy = epall
cache_time = 604800
[chunker]
type = chunker
remote = union:
chunk_size = 1Gi
A log from the command with the -vv
flag
Difficult to reproduce (but whenever bug occurs no error is reported) (I might add something later)
(Also I think bug is independent of union or b2 remotes. Trying to reproduce in local...)
Observed Result :
Only metadata file is renamed i.e.
<b2_key_id_1>:<b2_bucket_1>/a.ext
becomes <b2_key_id_1>:<b2_bucket_1>/b.ext
All chunks are NOT renamed:
<b2_key_id_1>:<b2_bucket_1>/a.ext.rclone_chunk.001
<b2_key_id_1>:<b2_bucket_1>/a.ext.rclone_chunk.002
<b2_key_id_1>:<b2_bucket_1>/a.ext.rclone_chunk.003
<b2_key_id_1>:<b2_bucket_1>/a.ext.rclone_chunk.004
<b2_key_id_1>:<b2_bucket_1>/a.ext.rclone_chunk.005
...
remain of the same name.
Expected Result :
All chunks should also be renamed along with the metadata file. i.e.
<b2_key_id_1>:<b2_bucket_1>/a.ext.rclone_chunk.001
-> <b2_key_id_1>:<b2_bucket_1>/b.ext.rclone_chunk.001
<b2_key_id_1>:<b2_bucket_1>/a.ext.rclone_chunk.002
-> <b2_key_id_1>:<b2_bucket_1>/b.ext.rclone_chunk.002
...