Slow download from google, comcast?

I’m notcing as of late that my speed from google drive to local is barely breaking 32Mbps. I’ve been seeing multiple posts around here with others having the same issue. Here is what I have tested and noticed

mounted cache
mount normally
rclone sync
rclone copy
set --transfers higher
played with vfs settings
mounted with either cache or regular and did an rsync
tried all the above with versions 1.36-current.

One thing I’ve noticed though is on another server I have remote (which is also 1Gbps) is it doesn’t seem to have this issue. I have gig from comcast at home and I’m wondering if comcast is somehow throttling? Not exactly sure, but through the browser I get full speeds.

I am having the exact same issue with comcast. At first I thought it was just me so I switched from plexdrive to rclone but am having the same results. its to a point where my plex media keeps buffering. Its a fairly recent thing as I have had not had this issue before. I also use a seedbox which does not have a throughput issue with gdrive. I have 300Mbps down at home. I am in the Seattle Area if we want to try corner this as well.

You’d probably have to run some network dumps/traces to see if the web downloads are using the same endpoints as rclone is.

I’m a Verizon FIOS gigabit customer using GD and I’m still pulling down 700-800 Mbs using rclone.

I’m in the chicagoland area as an FYI

how can I test this?

It depends on your technical savvy and what you have setup. I use ntopng and can see that route, my download from rclone:

Download from my GD Web.

I actually do better on the whole using rclone rather than grabbing from the web interface. It could be bad peering to the location that rclone is grabbing from. If there is a common thread of folks, you might be able to figure something in common.

You can run a copy like this:

felix@gemini:~$ rclone copy gcrypt:Radarr_Movies/"222 (2017)/222 (2017).mkv" /data -vv --stats 1s --dump headers
2018/08/14 14:28:01 DEBUG : rclone: Version "v1.42" starting with parameters ["rclone" "copy" "gcrypt:Radarr_Movies/222 (2017)/222 (2017).mkv" "/data" "-vv" "--stats" "1s" "--dump" "headers"]
2018/08/14 14:28:01 DEBUG : Using config file from "/home/felix/.rclone.conf"
2018/08/14 14:28:01 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2018/08/14 14:28:01 DEBUG : HTTP REQUEST (req 0xc42031cc00)
2018/08/14 14:28:01 DEBUG : GET /drive/v3/files?astuffle-apps.folder%27 HTTP/1.1
Host: www.googleapis.com
User-Agent: rclone/v1.42
Authorization: XXXX
Accept-Encoding: gzip

You can see both my copy and mount are going to www.googleapis.com, which for me peers to:

You can check which usually by:

felix@gemini:/$ host www.googleapis.com
www.googleapis.com is an alias for googleapis.l.google.com.
googleapis.l.google.com has address 172.217.7.234
googleapis.l.google.com has address 172.217.15.74
googleapis.l.google.com has address 172.217.7.170
googleapis.l.google.com has address 172.217.7.202
googleapis.l.google.com has address 172.217.15.106

Here are my results.

plex@server:~/gdrive/plexwrite/UHD Movies/Argo (2012)$ rclone copy plexwrite:"UHD Movies/Argo (2012)/Argo (2012).mkv" /tmp -vv --stats 1s --dump headers
2018/08/14 12:22:56 DEBUG : rclone: Version "v1.42" starting with parameters ["rclone" "copy" "plexwrite:UHD Movies/Argo (2012)/Argo (2012).mkv" "/tmp" "-vv" "--stats" "1s" "--dump" "headers"]
2018/08/14 12:22:56 DEBUG : Using config file from "/home/plex/.config/rclone/rclone.conf"
2018/08/14 12:22:56 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2018/08/14 12:22:56 DEBUG : HTTP REQUEST (req 0xc4201d8300)
2018/08/14 12:22:56 DEBUG : GET /drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%29%2CnextPageToken&pageSize=1000&q=trashed%3Dfalse+and+%27root%27+in+parents+and+name%3D%27Media%27+and+mimeType%3D%27application%2Fvnd.google-apps.folder%27 HTTP/1.1
Host: www.googleapis.com
User-Agent: rclone/v1.42
Authorization: XXXX
Accept-Encoding: gzip


2018/08/14 12:25:21 INFO  : 
Transferred:   104.996 MBytes (1.081 MBytes/s)
Errors:                 0
Checks:                 0
Transferred:            0
Elapsed time:     1m37.1s
Transferring:
 *                               Argo (2012).mkv:  0% /44.697G, 942.807k/s, 13h46m36s

 
plex@server:~/gdrive/plexwrite/UHD Movies/Argo (2012)$ host www.googleapis.com
www.googleapis.com is an alias for googleapis.l.google.com.
googleapis.l.google.com has address 216.58.217.42
googleapis.l.google.com has address 216.58.193.74
googleapis.l.google.com has address 172.217.3.202
googleapis.l.google.com has address 172.217.3.170
googleapis.l.google.com has IPv6 address 2607:f8b0:4005:802::200a


plex@server:~/gdrive/plexwrite/UHD Movies/Argo (2012)$ traceroute www.googleapis.com
traceroute to www.googleapis.com (x.x.x.x 30 hops max, 60 byte packets
 1  _gateway (192.168.2.1)  0.472 ms  0.446 ms  0.426 ms
 2  96.x.x.x (x.x.x.x)  14.175 ms  14.144 ms  14.112 ms
 3  po-102-rur202.bellevue.wa.seattle.comcast.net (68.87.207.193)  14.054 ms  14.057 ms  14.027 ms
 4  be-203-ar01.seattle.wa.seattle.comcast.net (69.139.164.169)  15.379 ms  15.324 ms  15.320 ms
 5  be-33650-cr01.seattle.wa.ibone.comcast.net (68.86.93.165)  16.021 ms  16.424 ms  15.955 ms
 6  be-10847-pe02.seattle.wa.ibone.comcast.net (68.86.86.226)  15.474 ms  12.878 ms  12.864 ms
 7  50.242.149.198 (50.242.149.198)  12.328 ms  13.531 ms  13.353 ms
 8  * * *
 9  209.85.243.8 (209.85.243.8)  13.294 ms 74.125.253.66 (74.125.253.66)  16.329 ms 108.170.233.158 (108.170.233.158)  16.269 ms
10  108.170.245.108 (108.170.245.108)  16.673 ms 108.170.245.107 (108.170.245.107)  16.793 ms 108.170.245.123 (108.170.245.123)  16.681 ms
11  72.14.233.116 (72.14.233.116)  29.977 ms 216.239.50.39 (216.239.50.39)  31.945 ms  16.325 ms
12  74.125.37.204 (74.125.37.204)  31.876 ms 216.239.46.208 (216.239.46.208)  23.149 ms 216.239.46.200 (216.239.46.200)  22.550 ms
13  209.85.252.207 (209.85.252.207)  37.482 ms 74.125.253.93 (74.125.253.93)  50.256 ms 209.85.252.207 (209.85.252.207)  37.440 ms
14  72.14.237.146 (72.14.237.146)  37.242 ms  36.986 ms  36.713 ms
15  108.170.242.241 (108.170.242.241)  36.900 ms  37.004 ms  36.868 ms
16  209.85.247.55 (209.85.247.55)  36.258 ms  36.124 ms  36.230 ms
17  sfo07s17-in-f74.1e100.net (172.217.6.74)  36.205 ms  35.999 ms  35.980 ms

I disovered by turning on ipv6 in my router that it starts at a higher bandwidth and dwindles down to the 1.5 again. odd.

so just tested a vpn service, that resolved the problem but I don’t get it. How?

You probably peer’ed to a different host.

so it does seem comcast is throttling the traffic then

I mean, it’s possible, but would be very odd for Comcast to throttle 443 traffic to google.

When I say bad peering, I mean that whatever route it is taking is slow. You seem to get around that by using a VPN to change the endpoint you were going to so it took a different route to your data.

Do you have any quotas or something on Comcast that slows things down after a certain peak or something?

Nope I have the unlimited plan with them. Again it’s a bit odd that browser=fine but literally any other method results in 30Mbps roughly. Honesty I’m not THAT tech savvy to setup what you did unfortunately.

Is there also a way to change the endpoint it uses if that is the case?

So update, I’ve done: Custom domains with dnsmasq following https://github.com/RMerl/asuswrt-merlin/wiki/Custom-domains-with-dnsmasq as I have a router with merlin firmware running. Still same results. Comcast has to be throttling somehow.

I don’t think they throttle anything. Few articles on the topic:

Well then I’m out of ideas

So the other thing you could try, is when you did the:

host www.googleapis.com
www.googleapis.com is an alias for googleapis.l.google.com.
googleapis.l.google.com has address 216.58.217.42
googleapis.l.google.com has address 216.58.193.74
googleapis.l.google.com has address 172.217.3.202
googleapis.l.google.com has address 172.217.3.170

You could try to manually put each one of those into your hosts file and do a test and see if one particular end point gives you a better response.

Here is another set of items that resolve near me but they’d probably be not close to you, but worth a try to as well.

host www.googleapis.com
www.googleapis.com is an alias for googleapis.l.google.com.
googleapis.l.google.com has address 172.217.3.42
googleapis.l.google.com has address 172.217.7.234
googleapis.l.google.com has address 172.217.8.10
googleapis.l.google.com has address 172.217.15.74
googleapis.l.google.com has address 172.217.7.170
googleapis.l.google.com has address 172.217.7.202
googleapis.l.google.com has address 216.58.218.234
googleapis.l.google.com has address 172.217.15.106

So an update on this, comcast is going to come out to my house tomorrow to take a look at the endpoints. I will report back with any updates on the matter.

An update on this

  1. Sorry for the delay
  2. The problem has been resolved using the following link even though I said it didn’t work, it did. I overlooked having my router set to use itself as a DNS point so now all my devices see the router as a DNS entry.

After setting one of the address that Animosity022 suggested I can see the problem has been resolved. Thanks again everyone.