I think part of the problem is that your "casual" browsing mount containing a lot of files generates a lot of traffic simply to pull remote's content again and again. By default --dir-cache-time is 5min meaning that after 5min given directory has to be re-read again.
GDrive is polling remote so you can safely set --dir-cache-time to "unlimited". Something like --dir-cache-time 9999h. Then add --vfs-refresh to get all data after mount is started (it happens in the background) and your experience should be much more smooth IMO. This way you will also cut number of API calls significantly - will help with throttling.
And yes, crucially, as mentioned, configure your own clinet_id. It is must. When done I suggest you re-create your remote to make sure that it is used.