Copying files from Crypt 1 to Crypt 2 using server side copy and not hit Google's daily API limits

Help and Support
1

What is the problem you are having with rclone?

I have a two crypts on two different Google Workspace accounts. Both the crypts have the same passkey and password 2 (salt) is empty.
I am trying to use the flag --server-side-across-configs to copy folders from crypt 1 to crypt 2 without using my local PC bandwidth and not hit the daily Google API limits.

However when I am testing out the flag I get the following Error:
Failed to copy: googleapi: Error 404: File not found: 1bv5ve5wUAQrdSRGxYnNPN7MMzbTOI6J8., notFound

Run the command 'rclone version' and share the full output of the command.

rclone v1.62.2

  • os/version: Microsoft Windows Server 2022 Datacenter 21H2 (64 bit)
  • os/kernel: 10.0.20348.1607 Build 20348.1607.1607 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.20.2
  • go/linking: static
  • go/tags: cmount
Yes

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy --server-side-across-configs CloudDrive:/"Folder A" FsCrypt: -vv

The rclone config contents with secrets removed.

[Workspace]
type = drive
client_id = xxxx
client_secret = xxxx
scope = drive
token = xxxx
team_drive = 

[CloudDrive]
type = crypt
remote = Workspace:/crypt
password = xxxx

[Fs]
type = drive
client_id = xxxx
client_secret =xxxx
scope = drive
token = xxxx
team_drive = 

[FsCrypt]
type = crypt
remote = Fs:Backups/Encrypted Backups
password = xxxx

A log from the command with the -vv flag

Failed to copy: googleapi: Error 404: File not found: 1yaLVwlAxRpSyQPf6r-TRbr_kqKrQYIE9., notFound
2

That means the source account doesn't have the right permissions in the destination to server side.

You can't skip the upload limit so you are stuck to 750GB per day upload.

3

Thanks for the reply.
Source is a crypt folder on one Google drive and destination is a crypt folder too on another Google drive.
Both the crypts have a same password.

How would I ensure that source have the right permissions to move/copy on the destination side?

Even if the above were to work with server side across configs flags, the upload limit would still come into play?
The whole reason I'm wanting to do this is to not use the API limits.
Thought with server side copy or move that would be the case? Am I wrong in thinking that?

4

You need to give the source account permissions on the destination. I don't use Google anymore so not sure exactly where it is. Content Editor or something.

Yes - Server side just means it doesn't use your bandwidth but all the upload and download limits are still in play.

A bit of semantics, but you are not hitting any API limits but you are hitting the 750GB per day upload quota. The API limits are huge.

5

So I added permissions by adding the user (from source Gdrive) to the crypt folder which is destination on the other GDrive.

If I use --Server-side flag then I still get Error 404.
But if I use either --drive-server-side or --crypt-server-side the debug shows files being transferrred but using the local bandwidth.

750GB upload limit per day would be applied even if you use "move"server side flag?

6

I may have it backwards as if you search the forums there are a few examples of the link I repost.

Correct. There's no way around that.

7

I got it to work , had to share the right permission for the two crypt mounts on Google Drive.

Now I am getting the error 403: rate_limit_exceeded for the copy function as I dont want to delete the files on Crypt 1 by using move command.

Using this command:
rclone copy --server-side-across-configs Crypt1: Crpyt2: --fast-list --tpslimit 10 --tpslimit-burst 10 -vv

But still getting rate limit exceeded.

8

Probably out of download and/or upload.

9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.