Google Drive speed is slow

Help and Support
1

Hey,

I am using rclone with an encrypted GDrive mount in order to backup my source codes and binaries. It’s an enterprise GSuite account with my own API key on my server. The server has two interfaces, but I am using rclone on the second, only 100/100 mbps network, because it’s very reliable, stable. However the rclone operations from/to the cloud are often really slow and not even near to 100 mbps. I must state that’s probably not rclone’s fault, but if you know a possible workaround, experience the same issue or just have something to tell me, please don’t hesitate to share. :slight_smile:

I did some benchmarks using the Vultr speedtest 1G binary (https://nj-us-ping.vultr.com/vultr.com.1000MB.bin) and the wget was really fine to my SSD:

vultr.com.1000MB 100%[==========>]   1000M  11.0MB/s    in 94s

So the network should be okay on my end. Then I tried to upload this to GDrive using a simple rclone copy -P command, but surprisingly that was even slower than using the mount function, having a fuse mount and using rsync -aP to copy the file there. Here you can see the rsync test’s end:

sending incremental file list
vultr.com.1000MB.bin
  1,048,576,000 100%    4.14MB/s    0:04:01 (xfr#1, to-chk=0/1)

The 4.14 is actually rare, it’s usually about 3.6 MB/s but even goes down to 2.8 MB/s for a longer period too.

Then the download is a bit more satisfying, but still closer to the half than the full speed of the link:

sending incremental file list
vultr.com.1000MB.bin
  1,048,576,000 100%    5.42MB/s    0:03:04 (xfr#1, to-chk=0/1)

However sometimes, for days it works blazingly fast without any issues. I wonder what can cause this… Google’s bandwidth is this poor?

2

What version of rclone are you using?

rclone version

Are you using your own API key?

The v3 API seems to cap at 20MB/s currently on a copy / sync command from the Google side.

felix@gemini:~$ rclone copy -P vultr.com.1000MB.bin gcrypt:
Transferred:   	 1000.244M / 1000.244 MBytes, 100%, 17.362 MBytes/s, ETA 0s
Errors:                 0
Checks:                 0 / 0, -
Transferred:            1 / 1, 100%
Elapsed time:       57.6s
3

The rclone version shouldn’t be that outdated, I updated a month ago or so:

rclone v1.45
- os/arch: linux/amd64
- go version: go1.11.2

And yes, as I mentioned I have my own API key.

Thank you for testing, that’s an incredible speed. :smiley:

4

Hmm. I don’t think Google has a bandwidth issue as that would be odd :slight_smile:

What’s your normal use case? Moving a single file each day or more files? I’d rather work to solve the issue than the 1 test file.

5

I zip the whole directory each 1 hour which is about 1.6 G and upload that to Drive through a mount and simply mv.

Sometimes it’s really fast, but often the whole stuff is slow like that. The CPU and the network on my end can’t be the issue nor the SSD which is in a perfect condition. So I got no clue really… :thinking:

And I mean Google can just limit the bw per transfer, but seems like it’s much bigger.

6

Try turning up your chunk size when you upload.

felix@gemini:~$ rclone copy -P vultr.com.1000MB.bin gcrypt: --drive-chunk-size 128M
Transferred:   	 1000.244M / 1000.244 MBytes, 100%, 36.576 MBytes/s, ETA 0s
Errors:                 0
Checks:                 0 / 0, -
Transferred:            1 / 1, 100%
Elapsed time:       27.3s

If you are just uploading bigger files.

7

With that, the download speed is amazing, actually reached the 10 MB/s.

But the upload is still the same:

Transferred:      163.723M / 1000 MBytes, 16%, 3.554 MBytes/s, ETA 3m55s

That’s the normal speed now. I was thinking about maybe it’s because of the encryption, so I sent it to a screen session, but the CPU usage was at 2%, and the load was 0.03. Plus like yesterday, the upload was 8 MB/s and it was stable… So it should work on full speed.

I have no idea.

8

Seems like this is similar: Speed to Gdrive

Is it a wise idea to increase the transfers?

9

If you are moving more than 1 file, yes, but if you are only transferring 1 file, having more files being transferred at a time wouldn’t matter.

--drive-chunk-size 128M

That’s only used when uploading a file to your GD.

 --drive-chunk-size SizeSuffix                Upload chunk size. Must a power of 2 >= 256k. (default 8M)

Where are you going to and from? Is everything US based or moving from one place to another? Is it a shared VPS or something dedicated?

My use case is a bit more controller as I have a dedicated server on gigabit FIOS that I control and can see all the traffic going in/out.

10

Everything is US based, yes.

The VPS itself is shared except the storage and the network is dedicated too.

11

Hmm. I remember seeing other posts about people complaining about speeds but might want to check with the VPS provider to see if they see anything.

I am not sure I see anything else as something looks to be throttling you on the VPS but not sure what offhand.

12

Is it faster with multiple files?

13

To be honest I can get the maximum speed from both upload and download if I am transfering files from the VPS to another using webdav. Only rclone+gdrive does that sometimes. And they are not on the same network. Different provider and location too.

And yes, it’s definitely faster with multiple files. If I have two copies of this 1 G file, the upload is exactly doubled.

14

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.