Server-side copy (sync) between to S3 buckets (different providers)

Help and Support
1

What is the problem you are having with rclone ?

I'm trying to sync two S3 buckets on different providers (OVH and Scaleway) but server-side copy doesn't seem to work.

Does it work only on buckets within the same provider (OVH > OVH) ?

Run the command 'rclone version' and share the full output of the command.

rclone v1.58.1

  • os/version: ubuntu 20.04 (64 bit)
  • os/kernel: 5.4.0-109-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.17.9
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

OVH S3 and SCALEWAY S3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync ovh:data scw:data --s3-storage-class GLACIER --fast-list --checksum

The rclone config contents with secrets removed.

[scw]
type = s3
provider = Scaleway
access_key_id =
secret_access_key =
region = fr-par
endpoint = s3.fr-par.scw.cloud
acl = private

[ovh]
type = s3
provider = Other
access_key_id =
secret_access_key =
region = de
endpoint = s3.de.cloud.ovh.net
acl = private

A log from the command with the -vv flag

2022/05/06 09:36:50 DEBUG : S3 bucket data: Waiting for transfers to finish
2022/05/06 09:36:56 DEBUG : data/chunks/3/3007/3007495_0_18715: md5 = dd7ebe0d9dfb47b199a08ffeb6164727 OK
2022/05/06 09:36:56 INFO  : data/chunks/3/3007/3007495_0_18715: Copied (new)
2022/05/06 09:36:56 DEBUG : data/chunks/3/3007/3007613_0_17167: md5 = 07f77f27edabfb7fc491dcfeadc84dce OK
2022/05/06 09:36:56 INFO  : data/chunks/3/3007/3007613_0_17167: Copied (new)
2022/05/06 09:36:56 DEBUG : data/chunks/3/3007/3007958_0_18618: md5 = de19f953c90167b302591396d01c9d2c OK
2022/05/06 09:36:56 INFO  : data/chunks/3/3007/3007958_0_18618: Copied (new)
2022/05/06 09:36:56 DEBUG : data/chunks/3/3007/3007349_0_7009: md5 = ee3a1ac073f718ee97b5d5e11681d5af OK
2022/05/06 09:36:56 INFO  : data/chunks/3/3007/3007349_0_7009: Copied (new)
2

I found this:

Server-side copies will only be attempted if the remote names are the same.

This answer my question but it is not very clear on docs. Let me improve that.

3

Not really just if the remote names are the same.

You can only server side copy generally if you are on the same provider.

Google Drive to Google Drive between remotes is fine as an example.

In your case, you are trying to server side from one provider to another which isn't a thing while they are both S3, they are different providers.

4

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.