GDrive uploads sometimes stall and require restart

Help and Support
1

From my VPS provider, my upload speeds to google can vary from 100MB/s to 5MB/s which I put down to congestion.

There is however a frequent occurrence where an upload will reach say 80-90% completion and then the speed will drop to 70-100KB/s.

The upload will finish eventually, but if I’m watching it, it is faster to restart the transfer, and it will zoom along at a good speed.

This occurs with simple command like rclone -v copy ./folder/ gdrive_crypt:folder/ and I am using my own API key.

Is this a known issue? Any rclone flags to ensure that rclone recognises the abnormal speed and restarts the transfer?

2

Are these big files that are being transferred in chunks?

It might be worth running with -vv to see what the debug log says about what is going on.

Also how are you measuring the speed? If you are using rclone itself then using the latest beta will give a more reliable speed measurement. It may be an artifact of the measurement…

There aren’t but there have been proposals for a --min-bandwidth flag.

3

a --min-bandwidth flag would be awesome.

I have been keeping logs of transfers and can’t see anything of note. No errors, or otherwise…

there is a lot of Sending chunk 167772160 length 8388608 and not much else.

I thought I had solved this by using --drive-chunk-size 256M which indeed improves speed, but sometimes a single file gets left behind with something like 1% /16.748G, 82.614k/s, 58h22m32s but there are no errors preceding this.

These are large files, being transferred a few at a time. 10-20GB typically.

apart from rclone, i do monitor in slurm and nethogs. nethogs shows the rclone process chugging along at a healthy 80KB/sec as I look at it now with this one file remaining…

4

Hmmm…

I think that must be the provider somehow…

We could implement a --min-bandwidth flag - can you please make a new issue on github about it - thanks!