Slow speeds downloading and uploading

Thanks. I’ve tried with the --async-read=false flag in v1.53 as suggested with very similar speeds but more stable.
The download time has gone down from 5m10.4s without the flag to 3m55.6s (with vfs-cache-mode writes and 3m50.6s with full) with the flag for the same file from the mount and max speed of 10.19MB/s (but far from the 35MB/s I get when copying from the remote directly). (Debug log)

rclone copy Movie.mkv ~/ -P
Transferred: 2.332G / 2.332 GBytes, 100%, 10.132 MBytes/s, ETA 0s
Transferred: 1 / 1, 100%
Elapsed time: 3m55.6s

                           rx         |       tx
--------------------------------------+------------------
  bytes                     2.45 GiB  |       18.48 MiB
--------------------------------------+------------------
          max          101.92 Mbit/s  |      920 kbit/s
      average           88.48 Mbit/s  |   651.49 kbit/s
          min              23 kbit/s  |       13 kbit/s
--------------------------------------+------------------
  packets                    1779661  |          270676
--------------------------------------+------------------
          max               8596 p/s  |        1594 p/s
      average               7477 p/s  |        1137 p/s
          min                 36 p/s  |           8 p/s
--------------------------------------+------------------
  time                  3.97 minutes

This is probably expected as v1.50.2 effectively does an rclone copy into the cache then serves the file from there.

Though it might be worth trying --vfs-cache-mode off just to check. I expect this will be the same as writes, but there are some corner cases which cause writes to use the same caching method as full.

The fact that the results are so similar for writes and full (which use completely different caching strategies) is probably pointing at a FUSE problem...

However we see FUSE running at full speed when copying the cached file so we know FUSE can run fast.

Hmm I have an idea, can you try the rclone copy direct from google drive with --multi-thread-streams 1 to make sure we disable multithread streaming. I wonder if each stream is limited to ~10MByte/s somehow. That would explain the differences (multi thread streaming isn't used from copying from local -> local which is what copying from the mount will appear like).

rclone copy remote:Movie.mkv /home/cere/ -P --multi-thread-streams 1

I think the problem might be sorted and you won't believe how... I don't even know what it has to do with it...
My dedicated server had its own DNS along with its own domain/search in /etc/resolv.conf, I've changed them for 1.1.1.1 and 1.0.0.1 and removed the search/domain.
After making that change the mount is working back to normal and speeds of copying directly from remote have doubled.

I would have never find out if it wasn't for a colleague that had the opposite issue (acceptable speeds on mount but very slow when copy from/to remote.

If you had the same results with rclone copy as rclone mount, you might have had bad peering which would point to DNS and you getting different results.

You had different results with both commands and they each would use the same IP that DNS would resolve to.

Based on the DNS, you do get different IPs and routes based on the results provided.

felix@guardian:~$ nslookup
ser> server 8.8.8.8
Default server: 8.8.8.8
Address: 8.8.8.8#53
> api.google.com
Server:         8.8.8.8
Address:        8.8.8.8#53

Non-authoritative answer:
api.google.com  canonical name = api.l.google.com.
Name:   api.l.google.com
Address: 172.217.11.4
> server 1.1.1.1
Default server: 1.1.1.1
Address: 1.1.1.1#53
> api.google.com
Server:         1.1.1.1
Address:        1.1.1.1#53

Non-authoritative answer:
api.google.com  canonical name = api.l.google.com.
Name:   api.l.google.com
Address: 172.217.9.228
>

Interesting...

If you have strange DNS then it is possible rclone won't be using the geographically closest Google endpoint.

Not sure how that explains the two different speeds as rclone mount and rclone copy do very similar API calls. I suspect that --multi-thread-streams might have something to do with it (see above).

Anyway - great it is working better!

I know... weird eh!? I almost format the server and start from scratch, thank god I didn't!

Anyway, thanks very much to everyone for helping me troubleshooting :slight_smile:

1 Like

I had exactly the same problem.
I changed my DNS from 8.8.8.8 to 1.1.1.1 and now the problem is gone. Which is really weird, as 8.8.8.8 is googles DNS, isn't it?
I still think that this has to be a routing problem with my provider rather than being a problem of googles DNS.
@cere: Any chance you are from Germany using internet via (tv)cable?

Best

Just a quick update: For me changing of the DNS did NOT solve the problem. The speed was/is changing, but often was limited to around 1.25 Mbytes. When i analyzed the connections, i saw that it connected to googles servers via IPv6. After binding rclone to ipv4 (via --bind) the problem went away. I started googling and i looks my provider (Vodafone cable in Germany) has currently problems with IPv6 connections). So temporarily the use of IPv4 solves the problem and in the long run i hope the provider fixes the issue.

Best

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.