Possible to sync between same cloud without downloading to local


As part of our automation we move files between aws, azure & google cloud buckets. In order to facilitate this, we are currently using rclone as it will give us uniform approach instead of working with various clis and programs. Even though everything is fine, we are observing slow speeds when files are being moved with in same cloud and so, would like to check if there is any option available in rclone that provides sync capabilities within same cloud without downloading files to local disk.

What you are asking for is a "server side copy". Yes rclone supports this as do s3, azureblob and gcs.

The main limitation is that you should use the same remote, so do

rclone copy s3:bucket1 s3:bucket2


rclone copy s3:bucket1 anothers3:bucket2

If you run with -v you can see in the logs whether the copies are done with server side copies.

1 Like

Thanks for the update. Yes, I am looking for between same backends. I ran the commands with verbose but didn't find any logs that shows whether files are being downloaded or being copied on server side.

Version 1.53.1

You should get something like this

$ rclone copy -v s3:rclone/file2.txt s3:rclone/file2a.txt
2020/12/30 10:55:43 INFO  : file2.txt: Copied (server-side copy)
2020/12/30 10:55:43 INFO  : 
Transferred:   	        26 / 26 Bytes, 100%, 86 Bytes/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:         0.6s

Note the server-side copy text.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.