Slow download from gdrive to local


#81

Any workaround or which latest version does not have 25MB/s cap?


#82

I don’t think there is a 25MB cap. I can still max out my gigabit link without an issue on a mount.


#83

Can back up @Animosity022’s statement. Easily able to max out my 2 gigabit link without any issues with 1.43.1 using the cache backend.


#84

I like you word “think”

github issue: https://github.com/ncw/rclone/issues/2573

1.43.1

copy --transfers=1 25MB/s capped

Transferred: 330.996M / 31.505 GBytes, 1%, 21.720 MBytes/s, ETA 24m30s
Errors: 0
Checks: 3 / 3, 100%
Transferred: 0 / 2, 0%
Elapsed time: 15.2s
Transferring:

  •                                G4D-V6.GHO:  1% /27.692G, 25.156M/s, 18m34s
    

2018/09/16 05:05:32 INFO :
Transferred: 375.246M / 31.505 GBytes, 1%, 21.729 MBytes/s, ETA 24m27s
Errors: 0
Checks: 3 / 3, 100%
Transferred: 0 / 2, 0%
Elapsed time: 17.2s
Transferring:

  •                                G4D-V6.GHO:  1% /27.692G, 24.736M/s, 18m51s
    

2018/09/16 05:05:34 INFO :
Transferred: 422.996M / 31.505 GBytes, 1%, 21.986 MBytes/s, ETA 24m8s
Errors: 0
Checks: 3 / 3, 100%
Transferred: 0 / 2, 0%
Elapsed time: 19.2s
Transferring:

  •                                G4D-V6.GHO:  1% /27.692G, 24.709M/s, 18m50s
    

copy --transfers=2 25MB/s capped x2

2018/09/16 05:03:41 INFO :
Transferred: 1.943G / 31.505 GBytes, 6%, 42.221 MBytes/s, ETA 11m56s
Errors: 0
Checks: 3 / 3, 100%
Transferred: 0 / 2, 0%
Elapsed time: 47.1s
Transferring:

  •                                G4D-V6.GHO:  3% /27.692G, 23.300M/s, 19m31s
    
  •                              GHOSTXPC.GHO: 24% /3.813G, 21.424M/s, 2m18s
    

2018/09/16 05:03:43 INFO :
Transferred: 2.038G / 31.505 GBytes, 6%, 42.469 MBytes/s, ETA 11m50s
Errors: 0
Checks: 3 / 3, 100%
Transferred: 0 / 2, 0%
Elapsed time: 49.1s
Transferring:

  •                                G4D-V6.GHO:  3% /27.692G, 23.264M/s, 19m31s
    
  •                              GHOSTXPC.GHO: 25% /3.813G, 21.863M/s, 2m13s
    

2018/09/16 05:03:45 INFO :
Transferred: 2.133G / 31.505 GBytes, 7%, 42.712 MBytes/s, ETA 11m44s
Errors: 0
Checks: 3 / 3, 100%
Transferred: 0 / 2, 0%
Elapsed time: 51.1s
Transferring:

  •                                G4D-V6.GHO:  4% /27.692G, 23.353M/s, 19m25s
    
  •                              GHOSTXPC.GHO: 26% /3.813G, 22.239M/s, 2m8s
    

copy --transfers=3 25MB/s capped x3

2018/09/16 05:08:14 INFO :
Transferred: 3.069G / 41.350 GBytes, 7%, 67.262 MBytes/s, ETA 9m42s
Errors: 0
Checks: 0 / 0, -
Transferred: 0 / 4, 0%
Elapsed time: 46.7s
Transferring:

  •                         G4D v6/G4D-V6.GHO:  3% /27.692G, 19.556M/s, 23m24s
    
  • … X64 [2016-05]/WIN7_SP1_X64_UP201605.iso: 20% /5.065G, 22.751M/s, 3m0s
  • …[2016-06]/2012.R2VL.ESD.ENU.June2016.iso: 25% /4.569G, 23.836M/s, 2m27s

2018/09/16 05:08:16 INFO :
Transferred: 3.173G / 41.350 GBytes, 8%, 66.682 MBytes/s, ETA 9m46s
Errors: 0
Checks: 0 / 0, -
Transferred: 0 / 4, 0%
Elapsed time: 48.7s
Transferring:

  •                         G4D v6/G4D-V6.GHO:  3% /27.692G, 20.055M/s, 22m47s
    
  • … X64 [2016-05]/WIN7_SP1_X64_UP201605.iso: 21% /5.065G, 22.756M/s, 2m58s
  • …[2016-06]/2012.R2VL.ESD.ENU.June2016.iso: 25% /4.569G, 23.492M/s, 2m27s

#85

Your backup is useless.
It does not mean that the cap is not exists.
mount command use multiple transfers, not single transfer


#86

I know there is not a 25MB cap. Better?

I replied to the issue as it seems to be related back to the copy command as the mount and copy both should use the v3 api.


#87

–vfs-read-chunk-size=0b

mount command read file by chunks (multiple transfers)

The 25MB/s limitations is existed for mount’s connection.


#88

No, you are not correct.

It’s a single transfer. Run the debug and look.


#89

then mount with “–vfs-read-chunk-size=0b” (disable chunks) only and show your rsync speed


#90

I was able to produce the 25MB/s cap as well. I added that to the bug with your information as well. Appreciate you stepping through the details!


#91

The mentioned workaround using the v2 API is now merged in the latest beta’s (since https://beta.rclone.org/v1.43-092-g03ea05b8-beta/).

Once I have the time to test more extensively when the error occurres, I will create a ticket in the Google tracker.
It could be related to the used region or something similar.


#92

Thanks, @B4dM4n, working great for both copy and mount.

For some others, You can’t tell me it’s not v2 when I am using the exact same settings besides v2 flag and it changes the speed.

Using v3 API, same mount settings except the v2 flag…
Copy:
image
Mount:
image

Using v2 API.
Copy:


Mount:
image


#93

Sadly, I’m now experiencing ~20Mbps cap even with the v2 api (download links).
I’ve noticed that around 2 weeks ago.
ps: when downloading via the official gui (drive.google.com) I easily saturate my link.

update: So far I’ve tested different versions of kernel, tried running on bare metal, tested mount vs copy, tried shaping mitigations (like user agent spoofing) but I was able to bring it down to bare http requests and have confirmed, that it’s something on the network (or peering to Google from my location).

So in my case it might not be software related :+1:


#94

What rclone version are you using?

What command are you using ?