Rclone speed on Google Drive only 4.1 MB/s

What is the problem you are having with rclone?

Slow uploads (I have 200 mbps)

What is your rclone version (output from rclone version)

1.52.3

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 2016 Server, 64 bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy --update --verbose --transfers 15 --checkers 15 --contimeout 60s --timeout 300s --retries 10 --low-level-retries 0 --stats 1s --drive-chunk-size=2048M D: gCrypt:/latest-copy

The rclone config contents with secrets removed.

[gdrive]
type = drive
client_id = 515556487875-faq8gbjde18mdmul4qv7jru0dbggaolr.apps.googleusercontent.com
client_secret = 
scope = drive
root_folder_id = 0AGhXF23f5XvHUk9PVA
token =
team_drive = 0AGhXF23f5XvHUk9PVA

[gCrypt]
type = crypt
remote = gdrive:/gCrypt
filename_encryption = obfuscate
directory_name_encryption = true
password = 
password2 = 


A log from the command with the -vv flag

2020/09/15 14:07:39 DEBUG : Using config file from "C:\\Users\\Administrator\\.config\\rclone\\rclone.conf"
2020/09/15 14:07:39 DEBUG : fs cache: renaming cache item "D:" to be canonical "//?/D:/"
2020/09/15 14:07:40 DEBUG : fs cache: renaming cache item "gdrive:/latest-copy" to be canonical "gdrive:latest-copy"
2020/09/15 14:07:40 ERROR : System Volume Information: failed to open directory "System Volume Information": open \\?\D:\System Volume Information: Access is denied.
2020/09/15 14:07:41 DEBUG : Google drive root 'latest-copy': Waiting for checks to finish
2020/09/15 14:07:41 DEBUG : Google drive root 'latest-copy': Waiting for transfers to finish

hello and welcome to the forum,

if you use the defaults, what is the speed?
rclone copy --update --verbose D: gCrypt:/latest-copy

uploading to gdrive can be very slow.
https://rclone.org/drive/#limitations

Transferred: 147.403M / 501.497 GBytes, 0%, 2.498 MBytes/s, ETA 2d9h5m47s
I am uploading 2 files, both over 100gb. The other 300gb are small. The files being uploaded when I copied this from the log are the 2 big ones.

Thanks!

Your --drive-chunk-size is very big. That could possibly affect things I'm not sure.

if i want to understand what to expect from a backend.
i create four files of randomized data sized of size 5GB.
then have rclone upload them.

also, you might want to read this post, where i solved a problem with a windows computer and slow upload to gdrive.

Ran rclone without the settings and it was half the speed, so I don't think that's it

How much memory on your server?
Can you remove the transfers/checkers and such as they don't add much value other than getting too many API hits per second.
Can you post a full log of the transfer as there's only a small snippet with no detail on the transfer times or anything else.

96gb. OK, I'll remove the checkers but last time I did that it 1/2ed the speed. OK

That seems odd if you are hitting OOM with 96GB.

What does OOM mean?

Out Of Memory͏͏͏͏͏͏‏

I think I cross posted somehow from a different topic so apologies for that.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.