Possible to sync between same cloud without downloading to local

[RaghavendraP]

Hi,

As part of our automation we move files between aws, azure & google cloud buckets. In order to facilitate this, we are currently using rclone as it will give us uniform approach instead of working with various clis and programs. Even though everything is fine, we are observing slow speeds when files are being moved with in same cloud and so, would like to check if there is any option available in rclone that provides sync capabilities within same cloud without downloading files to local disk.

[ncw (Nick Craig-Wood)]

What you are asking for is a "server side copy". Yes rclone supports this as do s3, azureblob and gcs.

The main limitation is that you should use the same remote, so do

rclone copy s3:bucket1 s3:bucket2

Not

rclone copy s3:bucket1 anothers3:bucket2

If you run with -v you can see in the logs whether the copies are done with server side copies.

[RaghavendraP]

Thanks for the update. Yes, I am looking for between same backends. I ran the commands with verbose but didn't find any logs that shows whether files are being downloaded or being copied on server side.

Version 1.53.1

[ncw (Nick Craig-Wood)]

You should get something like this

$ rclone copy -v s3:rclone/file2.txt s3:rclone/file2a.txt
2020/12/30 10:55:43 INFO  : file2.txt: Copied (server-side copy)
2020/12/30 10:55:43 INFO  : 
Transferred:   	        26 / 26 Bytes, 100%, 86 Bytes/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:         0.6s

Note the server-side copy text.