Weird Variation In Upload/Download time for Google Drive

Help and Support
1

Apologies for not using the scaffold but I wanted to keep things brief.
I've been running a few tests to try and optimize downloads and uploads through rclone and Google Drive via very quick 2MB (or precisely, 2000896 bytes) file uploads/downloads. I've recorded results for a few tests, do note that these tests were performed with the --fast-list, --ignore-checksum, and --drive-chunk-size=8M flags.

I'm currently running on a 50 Mbit Down, and 20 Mbit Up, ethernet connection to the internet:

Upload trial count: 46
Upload fastest time: 3.5 seconds
Upload slowest time: 16.78 seconds

Download trial count: 18
Download fastest time: 5.51 seconds
Download slowest time: 20.92 seconds

With my internet connection:
Ping to api.google.com
Low: 17.123 ms
Average: 19.272 ms
Max: 26.416
Packets: 22 (64 bytes)

Expected fastest upload time: 2000896 (filesize) /(20 (mbit) *125000 (convert to bytes) = 0.8004 seconds
Expected fastest download time: 2000896 (filesize) /(50 (mbit) *125000 (convert to bytes) = 0.3201 seconds

What I noticed with the --dump headers log. Is there seemed to be about a 2 second wait time between an HTTP request and a response. Does anyone know any way to optimize this? I might do more trials using my own client id. Let me know if you want the logs from the HTTPS dumps.

2

really should do that first, then run your tests.
imho, not sure the value of debugging your current setup as is.

please answer all the questions, so we can help you

3

Version Info:
rclone v1.66.0

  • os/version: darwin 13.6.3 (64 bit)
  • os/kernel: 22.6.0 (arm64)
  • os/type: darwin
  • os/arch: arm64 (ARMv8 compatible)
  • go/version: go1.22.1
  • go/linking: dynamic
  • go/tags: cmount

Cloud storage system:
Google Drive

Commands:
rclone copy /path/to/examplefolder/file_example_2MB.txt gdrive:/ --fast-list --ignore-checksum --drive-chunk-size=8M -vv

rclone config redacted output:
[gdrive]
type = drive
scope = drive
token = XXX
team_drive =
root_folder_id = XXX
client_id = XXX
client_secret = XXX

Log file:
log3_up.txt (1.9 MB)

I noticed the main delay is between the HTTP request and response, I would assume with a low enough ping and high upload speed that HTTP requests would take sub 100 ms to respond, not up to 1 second.

4

which transaction do you mean, please be specific.

5

I mean, each HTTP transaction seems to take about a second to respond. I thought transactions were meant to be fairly quick, about sub 100ms. Possibly longer for the actual download process but I thought any checks before that would be quicker.

6

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.