Google Drive Server Side Copy

Decided I need to copy/move some files from one Google drive account to another. I was looking with great interest at the server side copying option, https://rclone.org/drive/#drive-server-side-across-configs.

I am using:

rclone copy Media:Mops School:Mops -vvv --drive-server-side-across-configs is my command.

It is copying the files just fine, but it is copying them using local bandwidth, so it's not using server-side.

I see others have posted about similar titled (in the forum) issues, but not not quite the issue.

Thanks.

I don't see any obvious mistake here, but...

I have always been a little unsure about which of the two remotes the --drive-server-side-across-configs the flag will apply to in this syntax (or possibly both?)

Try this...
Open your rclone.conf and insert this at the bottom of both your Gdrive remotes:
server_side_across_configs = true

Then save the config and try the command again (you will not need the --drive-server-side-across-configs flag then, but it won't hurt either - just redundant).

Does that work better?
Note that the easiest way to check is probably to enable verbose output with the flag:
-v
and then each copied file should have (copied server-side) at the end of the line if it is working correctly.
Also check if there are any ERROR lines.

I also highly recommend you use the latest rclone version (currently 1.50)
Many users use old versions from repositories on Linux especially, and there have been a LOT of improvements to this in the last handful of versions. Easiest way to check your version is to run:
rclone version

Easiest way to get the latest version straight from rclone is

curl https://rclone.org/install.sh | sudo bash

Test and report back :slight_smile:

I had already tried that in my bug checking, but I do have 1.49.5, so I will update and see if that makes a difference. I missed seeing that update.

I updated, and used the same command line,and I had already added that flag to both, as I too am unsure.
The speed is about 871.085 kBytes/s, and the amount is so small, it seems like it is using local bandwidth, as I would certainly think Google servers copy a lot faster than that between themselves.

About all I can do is run the command in a safe mode with limited services, but I believe it will show the same thing. This cannot be sever-side.

Thoughts?

I am doing something similar, and seeing speeds of about 2MB/sec.
However, I can confirm that it is still using server side copy because the verbose output says so.

2020-01-03 19:26:05 INFO : Files/files/project1.txt: Copied (server side copy)

The directory I'm moving contains about 400 small text files, sized between a few k and 2MB. So I think the slow speeds are not the transfer itself, but rather the overall throughput due to latency grabbing the files from storage.

The throughput rclone shows with -P is actually an average. So, if my files are transferring at 100MB/sec, but between every file there is a pause of a few seconds, rclone would show the kind of numbers I'm witnessing at the moment.

Another way to see what I'm talking about is to run with -P

Transferred: 5.326G / 5.624 GBytes, 95%, 2.791 MBytes/s, ETA 1m16s
Errors: 0
Checks: 0 / 0, -
Transferred: 307 / 614, 50%
Elapsed time: 4m50.2s
Transferring:

  • Files/files/project1.txt: 0% /138.062k, 0/s, -
  • Files/files/project2.txt: 0% /216.468k, 0/s, -
  • Files/files/project3.txt: 0% /671.354k, 0/s, -
  • Files/files/project4.txt: 0% /160.347k, 0/s, -

NOTE: math won't work because this isn't the full output

The files being transferred always show 0% and 0/s because they transfer almost instantly, and are replaced by the path to the next file, which also shows 0% until it is replaced by the path to the next file, rinse repeat.

Might that explain what you're experiencing?

I'm going to piggyback this to say that I can not get SSC to work from a crypt team drive to another crypt team drive. That is the only time I'm seeing it not work.

I don't know why that would be different? But the same config allows SSC in all other instances (i.e. within the team drive and user-to-team drive, just not team drive to team drive)

Small files are the problem as it can only create a 2-3 files per second.

Going from crypt to another crypt, you can't server side as it has to encrypt it so it downloads it and uploads it again. Encryption is a rclone thing and not a Google thing so it has to flow throw rclone and cannot be done server side.

If the keys are the same for the two crypts then the best thing to do is to sync the underlying remotes.

1 Like

You need to share the folder with the destination account, otherwise server side copy wont work.

Ha, yeah that totally makes sense and I have no idea why I didn’t think of that. Thank you.

@ncw had the answer (of course), but these are both team drives I created and are only accessible by me. Essentially they are just folders that can be hidden so as not clutter up my main drive when using the web interface.

Just for clarity should I run in to this in the future though, are you saying share the folder not the entire drive? @Harry

my reply was for @mylogon not you, sorry about that.

You cannot share folders in Shared Drives. You can only share the entire drive :wink:

That said, the same rule applies. To be safe share both Shared Drives with both accounts.