Update:
Was checking my download speed every few days to see if it got better and it didn’t till yesterday. Suddenly I could download fullspeed (40-47mbytes) with rclone and drive file stream without any hiccups. Was happy and thought that maybe my provider fixed it or something. But then today same crappy download as always.
Then I thought really hard about what could have been different that day. And after I tried a few things I remembered that I uploaded a huge file that day that took almost all day (I usually don’t upload anything from my home internet connection). Tried uploading something today and it really worked. While uploading I have constant fullspeed download from gdrive.
It doesn’t matter what I upload, where I upload to, and from which computer in my local network I upload from. As long as I upload something I have stable download speed for gdrive.
I’m really confused now.
Probably something wrong with my router? But I still don’t understand why it only affects gdrive and no other downloadtraffic.