Moving data from Google Team drive, quota question

Help and Support
1

Hi,

Using rclone Version: rclone v1.48.0 and google drive

I have some files in a team drive that have been uploaded via a service account.

example remotes:

Main Google Drive remote: gDrive:
Service account remote: teamDrive:

so if I run something like this:
rclone move -vv --drive-server-side-across-configs teamDrive:data/ gDrive:data/

Then the data will appear in my main Google Drive, and I can see from -vv it shows as
Moved (server side)

so my question is, does this use any of the Google Drive upload quota or are there any limits to how much you can move this way?

I know a server side copy has limits, but this is a move not a copy.

(Using the google drive web app/site I have moved stuff into team drives without seeing a limit, but it would be easier for my workflow to have rclone do it.)

Tia

2

The Web App/UI has no limits to my knowledge.

Anything you do being server side or regular upload seems to be stuck with a 750GB daily limit.

3

thanks for the reply, although not the answer I was hoping for :wink:

as a move is really just changing info on where the data is located in your drive. I was kind of hoping this would not.

4

So in answer to my own question, I decided to try it.

I make no promises, so at your own risk if you try, but this is what I did:

I checked the size of a few directories:

rclone size --include='Mar*' --include="Sen*" teamDrive:data

Which gave me:

Total objects: 75 Total size: 980.024 GBytes (1052292352424 Bytes)

Then did a move:

rclone move -v --drive-server-side-across-configs --include='Mar*' --include="Sen*" teamDrive:data gDrive:testdir/data

That worked, all the data appeared in the testdir (as it was just a test, I moved it back using the web app)

So that should be over the 'quota'

Then I checked to see if both the remotes could still upload, so did a regular:

rclone copy gDrive:some_path/a_85_GB_folder teamDrive:some_other_path/a_85_GB_folder

and then a

rclone copy teamDrive:some_other_path/a_85_GB_folder gDrive:different_path/a_85_GB_folder

And both remotes completed their transfers

So that, at least to me looks like the move between remotes did not use any quota from either my main account or the service account.

And the added benefit of doing the move using rclone, is that you can see what's going on, as moving big folders in the web app, just says something like move started, you have no idea when it has actually finished the move.

5

The quota isn't an exact number nor documented anywhere. It's great you got it all copied.

6

Yea the server-side quota is a bit mysterious and not at all well documented currently.

I have a sneaking suspicion that it may actually be at least partially tied to the amount of files you transfer. I believe I read something to the effect that the server-side copy was effectively done using shared-files and then making a copy of the share or something like that? In that case other rules may apply than strict limit on the data-amount.

I generally tend to hit 200-300GB'ish before a server-side copy stalls out, but it seems to vary a lot. I transfer files of very varying sizes though. I notice you only had 75 items. This may (or may not) work better on a server-side copy than lets say many thousands of much smaller files totaling the same data-size.

More resarch is needed - because google is not open about these sorts of details. The only thing they have disclosed is "750GB/day upload".

7

Cheers for the reply,

I wonder if I should try with some bigger directories...

I moved a load of stuff into a team drive using the web app and have now changed my mind and want to move it back. I would prefer to do it with rclone as at least then I can see the progress, but it would be kind of annoying if it stopped part way through.

8

From my experience, there are no limits to server-side moves (have moved multiple terabytes of data a single day) and the regular 750 GB limit to server-side copies.

9

Oh right, I misread that and thanks for clarifying as well as a move has no quota since it is basically a rename.

A copy will eat up quota.

Thanks!

10

Just a note:
I use "Drive File Stream" and it seems to share the 750GB upload limit per day...
Once I had used up the limit using rclone,
Drive file stream didn't allow me to upload anything,
Just showed "upload queued" until the next day....

Isn't it for moving within the same remote,
But if we use 2 different gdrive remotes,
Then rclone will download and upload the files right?
Effectively, a move between 2 gdrive is a change of owner permission from 1st to 2nd, but rclone doesn't handle permissions,
Would love to see it happen someday though :blush:...

11

Server side moves have no limit even across remotes.

12

Thanks all, that was what I was hoping.

So now I am off to move it all back... and it seems to be working :slight_smile:

13

Yes you are right,
I just tested it...
Although by move I had meant was the data literally switches from one account to another (change of owner),
Rclone currently moves it from the source to Destination but the file owner remains the same...

14

Good info.

I should have specified my experiences are mostly based on server-side copy (in reference to earlier post). It makes a lot of sense that a move would be handled pure by permission change and thus not count against quota. I guess this should be fairly easy to test so I might do exactly that...

15

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.