Wasabi increase transfer speed

Hi

I have a high speed internet line and we are only seeing around 20mbytes/s on the upload to wasabi. is there any extra config i can add to rclone to help increase our upload speed?

Are you transferring big files or small files?

For big files experiment with

  --s3-upload-concurrency int          Concurrency for multipart uploads. (default 4)
  --s3-chunk-size SizeSuffix           Chunk size to use for uploading. (default 5M)

Note that these will use more RAM

Increasing --transfers is helpful too.

1 Like

Thank you for that info, they are large files we are transferring

with the below, is it a case of just trying different numbers or do they have to be multiples of something?

--s3-upload-concurrency int Concurrency for multipart uploads. (default 4)
--s3-chunk-size SizeSuffix Chunk size to use for uploading. (default 5M)

Just make them bigger :slight_smile: Note that memory usage is approx --transfers * --s3-upload-concurrency * --s3-chunk-size

hello and weclome to the forum,

i use wasabi and have many TB stored there.
i have a verizon fios gigabit connection and i saturate the connection all the time.

here is a screenshot of an upload to wasabi.

notice that there are two speeds listed.

  1. the overall speed for the rclone command - 66.037MB/s
  2. the actual speed for the current file being uploaded - 112.288MB/s
    as you can see they do not match, so be clear as to what speed you are writing about.

also, the overall speed will be lower, as for each upload, rclone has to calculate the md5sum of each file and that takes time.

what kind of internet connection do you have and what are the download speeds, upload speeds and ping times?

are you uploading a few large files or lots of small files?

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.