Direct cloud-to-cloud copy?


#1

For some source/destination combinations it may be possible to copy data directly from one cloud to another without copying the data through the machine running rclone. For example, it is possible to copy data from Google Cloud Storage to Azure by passing either a public URL or Signed URL to the Azure CopyBlob operation.

This has the potential to be significantly faster, particularly for large files, and when the computer running rclone has limited upload bandwidth. However, it can only be used in some situations and is incompatible with some rclone features (e.g. bwlimit, timeout). It would also require some significant internal API changes to expose methods for negotiating and initiating direct transfers.

Is this an approach that you might be interested in pursuing? Or is this outside the scope of what rclone aims to do?

Thanks for considering,
Kevin


#2

Your question similar to this

And ncw main dev of rclone says
“I don’t think you can do this with rclone, but I have seen people taking about sharing the files ( in Google drive), then copying them into your drive which you can do with the web interface”


#3

Thanks @alfred, I hadn’t seen the post you linked, thanks! That’s correct, this feature is not currently supported by rclone, but is something which is technically possible. I’m curious if the developers have any interest in adding this feature.


#4

Maybe by spending a message to @ncw


#5

Rclone has a mechanism for exposing public links to files, so with an appropriate flag that could be used as an input to the upload mechanism where it is supported.

I understand the making a public link function quite well, but the uploading from a blob I understand less well. Maybe the best thing to do would be to please make a new issue on github and if you could put links to the docs for cloud providers which do support uploading from a URL.