Rclone speed on Google Drive only 4.1 MB/s

[JIBSIL]

What is the problem you are having with rclone?

Slow uploads (I have 200 mbps)

What is your rclone version (output from rclone version)

1.52.3

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 2016 Server, 64 bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy --update --verbose --transfers 15 --checkers 15 --contimeout 60s --timeout 300s --retries 10 --low-level-retries 0 --stats 1s --drive-chunk-size=2048M D: gCrypt:/latest-copy

The rclone config contents with secrets removed.

[gdrive]
type = drive
client_id = 515556487875-faq8gbjde18mdmul4qv7jru0dbggaolr.apps.googleusercontent.com
client_secret = 
scope = drive
root_folder_id = 0AGhXF23f5XvHUk9PVA
token =
team_drive = 0AGhXF23f5XvHUk9PVA

[gCrypt]
type = crypt
remote = gdrive:/gCrypt
filename_encryption = obfuscate
directory_name_encryption = true
password = 
password2 = 

A log from the command with the -vv flag

2020/09/15 14:07:39 DEBUG : Using config file from "C:\\Users\\Administrator\\.config\\rclone\\rclone.conf"
2020/09/15 14:07:39 DEBUG : fs cache: renaming cache item "D:" to be canonical "//?/D:/"
2020/09/15 14:07:40 DEBUG : fs cache: renaming cache item "gdrive:/latest-copy" to be canonical "gdrive:latest-copy"
2020/09/15 14:07:40 ERROR : System Volume Information: failed to open directory "System Volume Information": open \\?\D:\System Volume Information: Access is denied.
2020/09/15 14:07:41 DEBUG : Google drive root 'latest-copy': Waiting for checks to finish
2020/09/15 14:07:41 DEBUG : Google drive root 'latest-copy': Waiting for transfers to finish

[asdffdsa (jojothehumanmonkey)]

hello and welcome to the forum,

if you use the defaults, what is the speed?
rclone copy --update --verbose D: gCrypt:/latest-copy

uploading to gdrive can be very slow.
https://rclone.org/drive/#limitations

[JIBSIL]

@asdffdsa

rclone copy --update --verbose D: gCrypt:/latest-copy

Transferred: 147.403M / 501.497 GBytes, 0%, 2.498 MBytes/s, ETA 2d9h5m47s
I am uploading 2 files, both over 100gb. The other 300gb are small. The files being uploaded when I copied this from the log are the 2 big ones.

Thanks!

[ncw (Nick Craig-Wood)]

Your --drive-chunk-size is very big. That could possibly affect things I'm not sure.

[asdffdsa (jojothehumanmonkey)]

if i want to understand what to expect from a backend.
i create four files of randomized data sized of size 5GB.
then have rclone upload them.

also, you might want to read this post, where i solved a problem with a windows computer and slow upload to gdrive.

Very slow speeds with Google Drive

windows has something called AutoTune and i know this can get disabled. i would run this on both your computer and brother's computer. netsh interface tcp show global To tweak AutoTune netsh int tcp set global autotuninglevel=normal and these are possible values disabled : uses a fixed value for the tcp receive window. Limits it to 64KB (limited at 65535). highlyrestricted : allows the receive window to grow beyond its default value, very conservatively restricted : somewhat restricted g…

[JIBSIL]

Ran rclone without the settings and it was half the speed, so I don't think that's it

[Animosity022]

How much memory on your server?
Can you remove the transfers/checkers and such as they don't add much value other than getting too many API hits per second.
Can you post a full log of the transfer as there's only a small snippet with no detail on the transfer times or anything else.

[JIBSIL]

96gb. OK, I'll remove the checkers but last time I did that it 1/2ed the speed. OK

[Animosity022]

That seems odd if you are hitting OOM with 96GB.

[JIBSIL]

What does OOM mean?

[asdffdsa (jojothehumanmonkey)]

Out Of Memory͏͏͏͏͏͏‏

[Animosity022]

I think I cross posted somehow from a different topic so apologies for that.