403 error messages with cached remote

I haven’t used the flag and still don’t and can pull 90MB/s from my mount. Not sure it’s accurate to say you need the flag to stream.

@Animosity022 Yes, i’ve just dug through a recent thread where you helped to investigate some related issues and your performance seemed to be better and more stable than average. So lucky you :grinning: .

I also found your thread about the vfs sweetspot but can’t find how i would setup my config / mount to relay on vfs instead of a cache remote. Can you point me somewhere here?

Hell, this is so annoying and brings down WAF to even negative values.

All my stuff is up to date on how I config and my whys are on my github:

I too had the 403 errors, after removing the V2 flag I’ve not received them again.

So remove that flag unless you absolutely need it!

Well, i’ll give it a try.

The thing is that my transfer rates had been so awfully low without the flag that the GD storage became unusable anyway.

So after regaining stable access to my GD content i wanted to figure out which setup will help me to not run into these 403 errors again but still have acceptable transfer rates.

I tried several mount options but avoiding the –drive-v2-download-min-size flag in each of them. It doesn’t make any difference here if i am mounting the cache, encrypted or even decrypted remote via rclone, rates seem to be pretty under acceptable values in every scenario. Just for fun, i tried using several VPN locations but with no noticable difference.

The interesting aspect seems to be that i can get rates around 20 to 50 MB/s in some situations but it seems to depend on the exact file i am trying to read from Google. And this is also not related to files that have been uploaded a long time ago or just a few days ago.

The only way to really read with the expected speed of line is to do it from the web, where i can nearly max out my ISP limit.

For not running into quota issues again, it would be helpful to know how to monitor the API hits i am causing by playing around with my settings. I found a graph telling me something about activity but i am not sure if this is the right place to look for it:

apihits

So maybe someone here can tell me if i am right and what values are most likely to trigger quota limits?

In the console, click on:

Google Drive APIS -> Quotas and change it to per 100 seconds.

Mine looks like:

I don’t use the cache and never hit an issue with speeds or 403s and can always max out my line copying from the mount which means playback is always fine.

Got it, thanks. Looks good from the values so far. I’ll keep an eye on it while tweaking my settings towards better performance.

Can you share what it was when you got a ban? Can you share what the error message was exactly when you got banned?

Sharing the information really helps as that’s why I’ve been asking and you don’t seem to want to post any details :frowning:

Sorry if I missed you asking for details at any point, of course i am willing to share details to solve these issues.

After stopping and restarting my mount with the --drive-v2-download-min-size 0 flag added again at 3:15 PM i started to receive bans again exactly at 4:30 PM:

2018/10/14 16:30:45 ERROR : worker-0 : object open failed 12404654080: open file failed: googleapi: got HTTP response code 403 with body:

The graph shows that much more hits occured when using the mount command without the V2 flag. But maybe i am misreading it and this only shows action for API V3 which would explain it:

I also don’t see where i triggered on of the limits here. The day count shows a value of about 15000.

You are missing the actual 403 and what’s it reporting so can’t tell what the actual error is so it’s still just a guessing game.

Need the line that outputs the actual 403 and what 403 error it is.

Oh, sorry for that. The screen reader on my notebook is a complete mess :-/.

2018/10/14 20:53:05 ERROR : worker-0 : object open failed 0: open file failed: googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded

So I’d file an issue on github with the v2 with cache as that definitely seems to be an issue.

I have had the same problem (403 downloadQuotaExceeded) for the last 2 days now. I was copying one gdrive to another and probably ran into the 750GB limit and started to get 403 bans. But even after 24 hours it was not lifted. I have not used the v2 with cache. My linux server uses rclone 1.41 but I have also tried with 1.43.1 without any success. This never happened to me and I have been using this config for the last year. I use my own auth and I am very far from hitting the API’s quotas from what I have seen on the graphs. Could it be something on gdrive side?

This is pretty similar to my case except i am using my own api credentials for onlythe last two days.

@BlindFish222 Today my gdrive started behaving normally again…after almost 3 days. Is it working again on your side?

@shenzo: I had it unmounted since Sunday and was too busy yesterday to give it another try. I’ve re-activated the mount right now and have to rebuild my plex media library once more as most items turned red. I’ll post the results here.

I can confirm that for the last five hours or so my library scan for plex is running without causing new 403 messages. Although performance in terms of transfer speed and scanning media is still pretty under average.

@shenzo: Are you currently adding the –drive-v2-download-min-size to your mount command? I will dare to put it back into mine after scan has finished without causing any new issues.

Well, the problem started happening again this morning…I only managed to copy 32GB on it yesterday and now the same 403 downloadQuotaExceeded error is displayed.

I had time to see a clear difference when adding the v2 flag though: transfer speeds (download TO the drive) for the same content went from 1.5MB/s to about 30MB/s. Does anybody have any idea what is happening?

I have just tried creating new OAuth credentials on the same gdrive that gave me downloadQuotaExceeded errors. Reconfigured rclone to use these credentials and now back to normal speed for me: about 25MB/s for a single transfer for upload and download. I am using the v2 flag over 1GB.