I'm trying to backup my encrypted gdrive to a new gsuite unencrypted. Since I started backing up today, I noticed it got DownloadquotaExceeded at the 2TB mark. Not even touching 10TB. Bear in mind I'm using SAs to bypass upload limit.
There are many factors to this,
Encrypt>unencrypt carries additional api hits causing it to blow.
I was using v2 to upload those files originally, now having the latest 1.51, its forced to v3 making additional api hit.
I would like to know definitively what's causing only 2TB download, These 2 are the only I had in mind. I will try to download them encrypted>encrypted and using back v1.39 tomorrow when the ban lifts.
Additional thoughts would be great. Would like to upload them asap.
The limit you are breaking here is the Download Quota, not anything related to uploads.
Nor is it an API limit, or there would be another type of error.
The download quota should still be 10TB/day as I understand, so it's unlikely you hit this.
There is however another limit involved, which I think may be the cause here:
If you download a spesific file very many times (hundreds of times in an hour or two) this might trigger a quota for that file. This should not take so long to reset. For shared files I believe this may trigger a bit faster, but I am not quite sure.
If you don't understand having actually downloaded any files that many times - a likely culprit is that you might have software that is doing it unbeknownst to you. For example, systems like Plex or similar (if their scan settings are not set appropriately for cloud-storage) may scan file contents - and often - to get various metadata that it can't get directly from listings. It may only read a handful of KB, but that will still register as a download for that file for the purposes of this quota. These types of scanning aren't a big deal on local storage, but it causes problems on cloud-backends. Both in terms of quotas and also general performance.
You can actually get spesific details of this from looking at the google API detailed logs. That should make it possible to see if anything like this is going haywire. If so - the solution is to fix the problem on the software-end (settings in many cases), as rclone is simply executing the requests that the software is making.
I've never hit this limit myself, so I have limited experience with how exactly it manifests.
I would definitely go check the google API website and look at your statistics for the last 24hrs.
While you can't see exactly how many GB was downloaded, you can see the amount of GET requests there have been ect. and that will give you a decent idea of what may be happening here..