Failed to copy: failed to open source object: BrokenMPU: Multipart upload has broken segment data

Help and Support
1

STOP and READ USE THIS TEMPLATE NO EXCEPTIONS - By not using this, you waste your time, our time and really hate puppies. Please remove these two lines and that will confirm you have read them.

I want to sync to remote s3 bucket on ovh. I used the following cmd :rclone sync src destination. i got always this error : status code: 409, request id :Failed to copy: failed to open source object: BrokenMPU: Multipart upload has broken segment data. I have tried different option concurrency disable checkum but not works.

What is the problem you are having with rclone?

Run the command 'rclone version' and share the full output of the command.

I'am using this version:
rclone v1.60.0

  • os/version: debian 12.4 (64 bit)

  • os/kernel: 5.15.0-89-generic (x86_64)

  • os/type: linux

  • os/arch: amd64

  • go/version: go1.19.2

  • go/linking: static

  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

I work with ovh cloud.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Paste command here

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

Paste config here

A log from the command that you were trying to run with the -vv flag

rclone sync -vv give the following output
2023/12/22 12:20:42 DEBUG : file.json: Size and modification time the same (differ by 0s, within tolerance 1ns)
Rclone sync -v give the output:
2023/12/22 12:22:41 ERROR : file.gz: Failed to copy: failed to open source object: BrokenMPU: Multipart upload has broken segment data.

Paste  log here
2
  1. Update your rclone to the latest version - v1.65 now

If it still does not fix your issue:

  1. Post content of your rclone.conf file (remove all secrets)
  2. Post full debug log not only fragment
3

I still have the same issue with the v1.65
here is my rclone.conf:
[ovh-s3-prd]

type = s3

provider = Other

env_auth = false

access_key_id =

secret_access_key =

endpoint = https://s3.gra.io.cloud.ovh.net/

acl = private

region = gra

location_constraint = gra

[ovh-s3-stg]

type = s3

provider = Other

env_auth = false

access_key_id =

secret_access_key =

endpoint = https://s3.gra.io.cloud.ovh.net

acl = private

region = gra

location_constraint = gra

All my log are the same as below :

status code: 409, request id: tx7f0b0ba808d84ba4b1ae0, host id: tx7f0b0ba808d84ba4b1ae0

2023/12/25 13:25:49 ERROR :imper.tif: Failed to copy: failed to open source object: BrokenMPU: Multipart upload has broken segment data.

status code: 409, request id: txfa26b4836460465aa7834, host id: txfa26b4836460465aa7834

2023/12/25 13:25:49 ERROR : zonefile.flv: Failed to copy: failed to open source object: BrokenMPU: Multipart upload has broken segment data.

status code: 409, request id: txbd0898d361e5436c955bc, host id: txbd0898d361e5436c955bc

2023/12/25 13:25:49 ERROR :BRIGA1.pdf: Failed to copy: failed to open source object: BrokenMPU: Multipart upload has broken segment data.

status code: 409, request id: txaf8fd1df8f7c4993a345c-00658982dc, host id: txaf8fd1df8f7c4993a345c-00658982dc

2023/12/25 13:25:49 ERROR : zonefile.flv: Failed to copy: failed to open source object: BrokenMPU: Multipart upload has broken segment data.

status code: 409, request id: tx79e0123cbc234615abc27, host id: tx79e0123cbc23
2023/12/25 13:25:49 ERROR : */zonefile.flv: Failed to copy: failed to open source object: BrokenMPU: Multipart upload has broken segment data.

status code: 409, request id: tx0cc9855d79ac44c887669-00658982dd, host id: tx0cc9855d79ac44c887669-00658982dd

2023/12/25 13:25:49 ERROR : 256111/zonefile.flv: Failed to copy: failed to open source object: BrokenMPU: Multipart upload has broken segment data.

4

I suspect that OVH S3 is not compatible with AWS S3 standard and it uses its own error codes. So unfortunately it won't work with rclone today.

5

I tried to look up what that error actually means...

It is defined here

And it is used here

I'm not sure what that is for exactly...

It would be worth giving the latest beta
a go to see if that helps.

I think this is probably some sort of incompatibility.

Possible work arounds

  • disable multipart uploads with --multi-thread-streams 0 and --s3-upload-cutoff 5G. This will only work for files < 5G in size
6

I got always the same error: ERROR : toto.gz: Failed to copy: failed to open source object: BrokenMPU: Multipart upload has broken segment data.

It has something to do with the last version maybe. isn't it ?

7

Did you try this workaround?

The latest version uses multipart uploads more than previous versions.

This is likely a provider bug / incompatibility though as rclone uses interfaces supported by AWS S3.