Sync AWS S3 bucket to Backblaze B2 Cloud Storage (diff)

What is the problem you are having with rclone?

We have a AWS S3 bucket with around 5TB of data. Looking to sync/mirror the entire contents (diff only) weekly from S3 to another provider Backblaze B2 Cloud Storage. Want to go directly from AWS S3 to Backblaze B2 (i.e. no intermediate). Is this possible? Thanks.

What is your rclone version (output from rclone version)

v1.55.1

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Linux Ubuntu 18.04

Which cloud storage system are you using? (eg Google Drive)

AWS S3
Backblaze B2 Cloud Storage

hello and welcome to the forum,

rclone sync will sync files that have have changed between aws and b2.
the first sync will copy all the files.
the second sync, if no files in aws have changed, no files will be copied to b2.

there is no concept of server-side sync between different providers.

if you want to avoid using an intermediate, you can rent a virtual machine from aws.
of course, that will cost you for the virtual machine and transfer fees.

Thanks for the reply @asdffdsa.

there is no concept of server-side sync between different providers.if you want to avoid using an intermediate, you can rent a virtual machine from aws.

No problem there, just need to know does it download files from the s3 bucket, store them on the AWS virtual machine and then upload to b2 cloud storage? I.E. the files will hit and be stored on the AWS virtual machine?

rclone on the vm would behave the same as rclone on a physical machine.

rclone does not store any file data on the hard drive.
for a given file, rclone will copy a chunk at a time from the source to dest.
and that chunk is stored in memory, not hard drive.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.