Terrible upload speed with rclone gdrive crypt

[lokiloko (animas)]

Rclone version tested : latest beta and latest stable

> $ rclone move -P /root/fin/ mjp69:/1 -vv --transfers=1 --checkers=1 --delete-empty-src-dirs --no-traverse --user-agent "gfghfghf"
> 2019/11/13 19:23:00 DEBUG : rclone: Version "v1.50.1-036-g2784c323-beta" starting with parameters ["rclone" "move" "-P" "/root/fin/" "mjp69:/1" "-vv" "--transfers=1" "--checkers=1" "--delete-empty-src-dirs" "--no-traverse" "--user-agent" "gfghfghf"]
> 2019/11/13 19:23:00 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
> 2019-11-13 19:23:11 INFO  : Encrypted drive 'mjp69:/1': Waiting for checks to finish
> 2019-11-13 19:23:11 INFO : Encrypted drive 'mjp69:/1': Waiting for transfers to finish
> 2019-11-13 19:23:11 DEBUG : a3qss4qqqdt98aba64scr8s9dl6m8plc6tgi9d9b4apkkudaiv3sl8bna78vfs47rhidmr6vmkaoofngu203e2s3kuqgtji45vodb28/ou2vcr55ej4t6suh3dcv6680qqn2sg6oao9vjgppdt0hoo3nne6vo6ntuh71b4kqulfbs3en4ut0sqgqunr5k3fbfua1l62ahtjb1jg: Sending chunk 0 length 8388608
> Transferred:        4.470M / 6.916 GBytes, 0%, 14.302 kBytes/s, ETA 5d20h45m45s
> Errors:                 0
> Checks:                 0 / 1, 0%
> Transferred:            0 / 4, 0%
> Elapsed time:       5m20s
> Checking:
>  * File.mp4:  0% /1.518G, 14.560k/s, 30h16m40s
> Transferring:
>  * File.mp4:  0% /1.518G, 14.560k/s, 30h16m40s

Here is my setup

rclone config
Current remotes:

Name                 Type
====                 ====
jp69                 drive
khe                  drive
mjp69                crypt
mkhe                 crypt
tvkhe                crypt


[jp69]
type = drive
client_id = 1deleted.googleusercontent.com
client_secret = mdeleted3
scope = drive
token = {"access_token":"deletedw","token_type":"Bearer","refresh_token":"1deletedc","expiry":"2019-11-13T20:15:35.989769575-05:00"}
team_drive = 0deletedA

[mjp69]
type = crypt
remote = jp69:/m
filename_encryption = standard
directory_name_encryption = true
password = XdeletedI
password2 = Kdeletedg

I'm trying to rclone move from local to mjp69 (gdrive-crypt) which goes to jp69 gdrive shared drive (team drive) as you see in my settings. However in the past few days I have terrible speeds as above.

Here is the list of things I have tried:

1. Using different server, IP, ISP
2. Using different client ID, email address.
3. Using different rclone flags, rclone --user agent
4. Using different shared drive( team drive)
5. Using different version of stable and beta of rclone.

All results the same terrible speed.

My OAuth consent screen is not verified by Google. and not published, is this the reason for this rate limit ?

I noticed that every time I have this problem, a pattern appears in metrics dashboard. The upload speed starts very high normal speed maxing out my upload speed, then spikes of 403 errors for drive.files.get operation , then my rclone upload speed is down to a terrible speed for weeks....
Here is example of that using latest stable rclone.

I'm well below my quota and I don't use Plex. So whats the problem and how can I solve this?

[Animosity022]

What OS are you running?

Can you let it run a little longer as well?

The flags you have are limiting the speeds too. You can remove checkers and transfers and just use the defaults.

You can add "--fast-list" and you can add " -drive-chunk-size" and make that something like 64M or 128M and see if that helps.

[thestigma]

The issue is clearly that you are stalling out on loads of 403 errors according to your API dashboard images.

403's happen primarily due to one of these things:

(1) Temporary API overload. A Gdrive can handle 1000requests/100sec. If you wanted to test this to be safe you could use --tpslimit 9 and see if it made any difference, but I doubt it. In this case you are getting far more 403 errors than is normal and with transfers 1 / checkers 1 you should never even get close to the API limit anyway. The defaults don't really stress the API much normally. Rclone also integrates a request-pacer for drive that scientifically is supposed to keep performance from dropping due to API-spam. It monitors requests and throttles back if the server starts giving you errors.

(2) You've hit the 750GB/day upload limit. At that point further uploads won't be possible until the next morning (the reset point varies a bit but generally during the night). Downloads should keep working though - which is weird when your errors are on GET... You do have a download limit, but it is 10TB/day so most users can never even reach that if they tried. You don't use Plex with autoscanning or something similar right?