Rclone mount random slow speeds

yes. just edit the script and change the instances in which it refers to KiB and MiB for KB and MB, respectively.

this is very much a backend issue. config-wise, there's currently not much that can be done via rclone to remedy the issue. the problem is that if your request ends up with the address of a slow server, you will be stuck with it and then download of 1GB+ files will take way too long finish. so far, the best solution that has been proposed is to test servers ahead of time and pre-select the ones that pass a performance criterion.

Thanks yeah I tried that didn’t work- I’ll send through the info you need in the GH. Thanks for your help.

What speeds are everyone experiencing? I’m noticing a limit of around 2.5Mb/s or around 20mbps.

So smaller files seem to play/transcode fine- but larger files are an issue. Plex buffers either direct play or transcoding.

I’m surprised there’s no option to multi thread on mounts? I know you can on the copy command.

https://forum.rclone.org/t/rclone-1-59-0-release/31808/11

that's what I've been doing as well. script-wise, I think it is better to test the current host first to make sure it is actually necessary to run the more comprehensive tests (from the set of IPs returned by dig).

same results here in south america.

Well, turning mount into multithread as rclone copy, could be the solution, since rclone copy is performing much better than mount at this moment, as it uses multiple of the routes, no matter if they are quick or not.

Some pretty poor performance coming from the US. 3 of the 8 endpoints provide sufficient speeds for 4k playback.

Couple of quick questions out of curiosity:

  • Does RClone only use the whitelisted IP's added to the hosts file?
  • Do whitelisted IP's get deleted if they test poorly next time the script is run?
  • If an IP is removed from the hosts file, will it impact any active streams running through that endpoint?

******** Google Endpoint Scanner (GES) ********


[GES] [INFO] The application started on Tue Jul 12 13:35:34 AEST 2022.
[GES] [INFO] Hosts backup file found. Restoring it.
[GES] [INFO] Please wait. Downloading the test file from 142.250.189.170...
[GES] [INFO] Parsing connection with 142.250.189.170.
[GES] [INFO] 212.7 MBytes. Above criterion endpoint. Whitelisting IP '142.250.189.170'.
[GES] [INFO] Please wait. Downloading the test file from 142.251.46.234...
[GES] [INFO] Parsing connection with 142.251.46.234.
[GES] [INFO] 5.097 MBytes. Below criterion endpoint. Blacklisting IP '142.251.46.234'.
[GES] [INFO] Please wait. Downloading the test file from 142.251.214.138...
[GES] [INFO] Parsing connection with 142.251.214.138.
[GES] [INFO] 5.290 MBytes. Below criterion endpoint. Blacklisting IP '142.251.214.138'.
[GES] [INFO] Please wait. Downloading the test file from 172.217.164.106...
[GES] [INFO] Parsing connection with 172.217.164.106.
[GES] [INFO] 225.3 MBytes. Above criterion endpoint. Whitelisting IP '172.217.164.106'.
[GES] [INFO] Please wait. Downloading the test file from 142.250.189.234...
[GES] [INFO] Parsing connection with 142.250.189.234.
[GES] [INFO] 180.4 MBytes. Above criterion endpoint. Whitelisting IP '142.250.189.234'.
[GES] [INFO] Please wait. Downloading the test file from 142.250.191.42...
[GES] [INFO] Parsing connection with 142.250.191.42.
[GES] [INFO] 4.355 MBytes. Below criterion endpoint. Blacklisting IP '142.250.191.42'.
[GES] [INFO] Please wait. Downloading the test file from 142.251.46.170...
[GES] [INFO] Parsing connection with 142.251.46.170.
[GES] [INFO] 4.919 MBytes. Below criterion endpoint. Blacklisting IP '142.251.46.170'.
[GES] [INFO] Please wait. Downloading the test file from 142.251.46.202...
[GES] [INFO] Parsing connection with 142.251.46.202.
[GES] [INFO] 4.878 MBytes. Below criterion endpoint. Blacklisting IP '142.251.46.202'.
[GES] [INFO] The whitelisted IP '142.250.189.234' will be added to the hosts file.
[GES] [INFO] The whitelisted IP '142.250.189.170' will be added to the hosts file.
[GES] [INFO] The whitelisted IP '172.217.164.106' will be added to the hosts file.

rclone uses https://www.googleapis.com to interface with the gdrive backend. by default, your machine will first check the hosts file for an address that matches www.googleapis.com and will return the first address that matches it (e.g., 142.250.189.170).

yes. I made a change to mine yesterday that allows running a precheck (USE_PRECHECK) to ensure that the current host is bad before running the more comprehensive tests (from the several ip candidates returned via dig). this reduces bandwidth usage and allows you to run the script more frequently, which is something that I've been doing for the past few days.

no. it will impact future requests to resolve www.googleapis.com.

If anybody has the skill to translate the script to a Windows batch, it would be greatly appreciated.

Beautiful thanks. If you have multiple whitelisted address it can return any one of these?

Excellent- and this works with multiple whitelisted addresses?

You mention “current host”, I thought the host changed every seperate connection attempt and uses whatever address it can? (Limited to what is in the host file)

Perfect!

Sorry for the 21 questions just trying to better understand how rclone works with these addresses.

Thank you for the script- I’ve already noticed a substantial improvement in streaming performance with it running every 4 hours.

Apart from the Windows script request (if anyone is capable of creating it), it would be good for the scripts to allow for IPs not returned from dig/nslookup. From my country/city I get a fixed set of IPs but I've discovered that IPs from other countries in Europe provide, currently, much better performance, for instance.

PS
I am trying to escalate this with Google customer support.

by default, it returns only the first match (starting from the beginning of the file until eof). anything fancier than that requires an specialized application (e.g., dnsmasq).

no. it works with the first match in the hosts file, per my previous comment.

I agree but this has not been implemented yet. one idea is for some of us to sponsor this implementation, which would justify bumping its priority dev-wise.

So rclone only ever uses the first whitelisted address in the host file- for all connections?

Interesting…

Sorry I don't. But if you feel like learning some Linux. Windows has a inbuilt Ubuntu VM you can play with. You'll have to manually update your Windows hosts file with the chosen IP address. But at least the script will run and do the testing for you (assuming you installed rclone).

Considering how fast foreign servers are working, particularly Singaporean ones for me. I've updated my script a little to accept manually included IPs that might not have been found by the local dig.

Create a file called .whitelist-apis in the same folder, and fill it in with some IPs that you like the look of. 1 IP per line. No extra writing please.

So what happens is dig will find a list of IPs that Google gives out for your region.
Then anything in the .whitelist-apis will be added into the mix.
Then anything in the .blacklist-apis will be removed from the apis list. This includes any whitelisted IPs.
Then the script tests the APIs that are left, and any that have turned bad will be added to the .blacklist-apis file.

Also from South Africa and thought it was ISP glad I am not alone on this.

I was thinking about this yesterday. So I would basically install rclone in the virtual machine, transfer the rclone.conf file in order for the VM installation to use it and run the script. The result would be copied to the hosts file in the VM and I would need to manually adjust it for the Windows hosts file, is that it?

download files slow speed - Google Drive Community
this is trending in support community but still they are not giving any attention to this and they are one of the biggest tech gaint in the world, forgot about solving this they don't even reply to this trending problem in community. form my point of view they are just avoiding it.

The guy starts the topic with "i have shared drive to download 4k movies and web series". Of course that's not gonna get the same attention, if any, as a legit business ticket. Unfortunate, but understandable.

1 Like

Yep, exactly that.
Mine and cg0's scripts announce what they're doing and what IPs have what speeds so you'll have a good idea of which IPs are the good ones.

How would I run this if I run rclone in a docker container that has a different network from the host? And would I have to restart the container every time the ip's update?