--drive-server-side-across-configs don´t works with Encrypted Remotes

What is the problem you are having with rclone?

Hello Everyone

I'm tryin to copy some media files from a team drive to a encrypted Remote in other team drive.

I use: --drive-server-side-across-configs

That works great in other scenarios but when I try to copy files to a encrypted remote that uses my local bandwidth, and I don't know why

I think that is not the quota, because I can use drive server side across in other remotes with no problem.

Thanks!

What is your rclone version (output from rclone version)

rclone 1.51

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10, 64 bits

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy --drive-server-side-across-configs EvenSpace:/"folder" MyDrive: -P

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=989049894242, userRateLimitExceeded)
2020-02-19 13:24:48 DEBUG : pacer: Rate limited, increasing sleep to 1.925325707s
2020-02-19 13:24:48 DEBUG : pacer: Reducing sleep to 0s
2020-02-19 13:24:48 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=989049894242, userRateLimitExceeded)

hello and welcome to the forum,

the reason for local bandwidth is that it is the rclone program, which is running on your local computer, that does the encryption.

  1. rclone downloads a chunk of the unencrypted file.
  2. rclone encrypts the chunk
  3. rclone uploads the chunk to the encrypted remote.
  4. goto step 1 until entire file is processed.

i do not use gdrive but i think that other users use a google virtual machine to get around the quota.

someone who uses google can share more details about that.

1 Like

You can't use server side copies with encryption as the encryption feature is handled by rclone so it has to download it, encrypt it and upload it again.

If you want to make identical copies of things, you can use server side copies.

1 Like

you can copy the underlying remotes using server side copy

1 Like

This is a known issue.

It can be pretty easily fixed by simply making the copy command use the direct remotes (ie. copying the encrypted files directly rather than decrypting then re-encrypting them - which must necessarily happen on the local PC).

For example if you have these remotes:
Gdrive1: (Drive remote)
Gcrypt1: (Crypt remote on Gdrive1:\crypt)
Gdrive2: (Drive remote)
Gcrypt2: (Crypt remote on Gdrive2:\crypt)

If you try:
rclone copy Gcrypt1: Gcrypt2: then server-side will not be possible because you have to decrypt and recrypt.
This will work fine with server-side though:
rclone copy Gdrive1:\crypt Gdrive2:\crypt

But there is one limitation you need to be aware of with this - the encryption settings for Gdrcypt1 and Gcrypt2 remotes must be the same. If you must re-encrypt with different keys or settings then server-side will be impossible and this is not something rclone can fix with improvements.

1 Like

really appreciate everyone's desire to help, I just asked one question and many answers that teach me. Regards!

1st Question:
What happens if I copy a file to another folder within the same crypt[gdrive] remote ?

Let's say to copy a file from Gcrypt1:folderA to Gcrypt1:folderB ? Is this possible with server sided copy? If so do I need to add "drive-server-side-across-configs" to the Gdrive1 or Gdrive2 config?

If not currently possible, it would be not to hard to implement I would assume?

2nd Question:
Now a more difficult case: Suppose I have the same encryption settings on Gcrypt1: and Gcrypt2:
I understand its not possible to copy from Gcrypt1:folderA to Gcrypt2:folderB

As you explained rclone can't do this yet and unfortunately your workaroud wouldn't work for me because I don't want to copy the whole remote and I don't know the encrypted name of the file I need.

In theory rclone could do it, although it would be not easy. First rclone would need to understand that both of the crypt remotes are inside a supported gdrive remote and that the encryption settings match (that information may be supplied by the user). Then rclone would need to identify the underlying encrypted file/folder names and finally copy the encrypted files/folders from one gdrive remote to another server-sided.

So basically I would say It could be possible and the individual steps themselves are not that complicated, but there are a lot of them and things could probably go wrong somewhere along the way ...?

Looking forward to hear your thoughts!

you could do a rclone mount for each crypted remote.
then use your favorite file manager to copy/move the files.

I think I misunderstood.

I was actually talking about rclone mounts not the rclone copy command. I just happen to see that the OP talked about the rclone copy command in this thread. Does server sided copy even work with mounts?

for moving files within the same encrypted mount, server side copy does work

for moving files between two encrypted mounts, server side copy does not work

But only if you use rclone copy - if you just use explorer to copy things in a mount, then rclone will definitely be downloading and uploading files.

not sure what you mean?

what i wrote seems to be correct.

if i move a file from one crypted remote to another crypted remote, both on wasabi, i get

2020/04/11 13:40:05 INFO  : test/test.txt: Renamed in cache
2020/04/11 13:40:05 INFO  : test/test.txt: Copied (server side copy)
2020/04/11 13:40:05 INFO  : test/test.txt: Deleted

It is perfectly correct. I thought the OP might have been copying files using Windows Explorer for example rather than rclone copy.

both those crypted remotes were mounted on my windows computer.
i used double commander to move that file

Moving will do a server side move, copying won't though.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.