Gdrive extremely slow when uploading a lot of small files


#1

When uploading a lot of small files, Gdrive is extremely slow (e.g. 2 kbytes/s) whereas when I upload a movie file, I am able to reach 2 Mbytes/s).

I tried with my own client ID as well and it didn’t improve things. I also played a bit with chunk size, transfers and checkers settings with no success.

Someone claimed that Gdrive rates lmit new file creation to about 3/s, is that accurate?

I was wondering if there’s any way to improve the upload speed when uploading a lot of small files.

Alternatively, are there similar cloud providers which do not have this issue? I played around with S3 but the lack of fast move/delete/rename operations is very frustrating.

Thanks.


Uploading to GDrive, any way to speed up upload of many small files?
#2

2 per second is the figure I have in mind, and yes it seems to be a real limit.

I haven’t found one yet :frowning:

Do you mean in rclone mount? In the latest beta rclone can mv files on s3 now!


#3

I know for most cases it’s not ideal, but I found if you add the small files to a .rar first then upload the .rar it is a lot faster.


closed #4