Gdrive extremely slow when uploading a lot of small files

[olalonde (Oli Lalonde)]

When uploading a lot of small files, Gdrive is extremely slow (e.g. 2 kbytes/s) whereas when I upload a movie file, I am able to reach 2 Mbytes/s).

I tried with my own client ID as well and it didn’t improve things. I also played a bit with chunk size, transfers and checkers settings with no success.

Someone claimed that Gdrive rates lmit new file creation to about 3/s, is that accurate?

I was wondering if there’s any way to improve the upload speed when uploading a lot of small files.

Alternatively, are there similar cloud providers which do not have this issue? I played around with S3 but the lack of fast move/delete/rename operations is very frustrating.

Thanks.

[ncw (Nick Craig-Wood)]

Someone claimed that Gdrive rates lmit new file creation to about 3/s, is that accurate?

2 per second is the figure I have in mind, and yes it seems to be a real limit.

I was wondering if there’s any way to improve the upload speed when uploading a lot of small files.

I haven’t found one yet

Alternatively, are there similar cloud providers which do not have this issue? I played around with S3 but the lack of fast move/delete/rename operations is very frustrating.

Do you mean in rclone mount? In the latest beta rclone can mv files on s3 now!

[CTOverton]

I know for most cases it’s not ideal, but I found if you add the small files to a .rar first then upload the .rar it is a lot faster.