Rclone upload speed is very slow with vultr object storage

STOP and READ USE THIS TEMPLATE NO EXCEPTIONS - By not using this, you waste your time, our time and really hate puppies. Please remove these two lines and that will confirm you have read them.

What is the problem you are having with rclone?

Rclone is giving very slow speed while uploading a 500MB file to Vultr object storage

Run the command 'rclone version' and share the full output of the command.

rclone v1.50.2

Nope

Which cloud storage system are you using? (eg Google Drive)

Vultr Object Storage

The command you were trying to run (eg rclone copy /tmp remote:tmp)

 rclone --progress copy /home/computerman/vultr-sgp1/secure.node.rdb vultr-sgp1:logo --drive-chunk-size=512M

The rclone config contents with secrets removed.


[vultr-sgp1]
type = s3
provider = Vultr
env_auth = false
access_key_id = xxxx
secret_access_key = xxxx
region =
endpoint = https://sgp1.vultrobjects.com
location_constraint =
acl = public-read
server_side_encryption =
storage_class =

A log from the command with the -vv flag

2023/05/24 00:47:54 DEBUG : rclone: Version "v1.50.2" starting with parameters ["rclone" "--progress" "copy" "/home/computerman/vultr-sgp1/secure.node.rdb" "vultr-sgp1:logo" "--drive-chunk-size=512M" "-vv"]
2023/05/24 00:47:54 DEBUG : Using config file from "/home/computerman/.config/rclone/rclone.conf"
2023-05-24 00:47:55 DEBUG : secure.node.rdb: Need to transfer - File not found at Destination
2023-05-24 00:47:55 INFO  : S3 bucket logo: Bucket "logo" created with ACL "public-read"
Transferred:           45M / 554.911 MBytes, 8%, 1.285 MBytes/s, ETA 6m36s
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 1, 0%
Elapsed time:         35s
Transferring:
 *                               secure.node.rdb:  8% /554.911M, 857.554k/s, 10m8s^C

would you mind to repeat this test with latest rclone version?

you use v1.50.2 from 2020... it is now v1.62.2

the easiest way to upgrade is to run:

rclone selfupdate

I tried to run that command but it is failing with the following message :
2023/05/24 06:36:44 Fatal error: unknown command "selfupdate" for "rclone"

you are right - it was only introduced in v1.55 of rclone.. my fault

Then just download the latest one from here:

With the latest version start off using rclone defaults and no extra flags, to establish baseline performance.
If that does not saturate your internet connection, then start tweaking flags and values.

I would recommend:

--s3-upload-concurrency and --s3-chunk-size

e.g.:

 rclone --progress copy /path/to/file vultr-sgp1:logo --s3-upload-concurrency 160 --s3-chunk-size 64M
 rclone --progress copy /path/to/file vultr-sgp1:logo --s3-upload-concurrency 80 --s3-chunk-size 64M
 rclone --progress copy /path/to/file vultr-sgp1:logo --s3-upload-concurrency 40 --s3-chunk-size 64M

What speed were you expecting?

You are getting 1.285 MBytes/s which is about 10 Mbit/s upload.

hi,

  • can you post the result of a speedtest?

  • provider = Vultr is not correct, need to use provider = Other

  • --drive-chunk-size=512M is for gdrive, not S3.

  • did a quick test on vultr and wasabi, which is a s3 clone using a very simple command, using defautls
    rclone copy .\file.ext remote: -vv --progress
    wasabi - Transferred: 719.438 MiB / 719.438 MiB, 100%, 41.479 MiB/s, ETA 0s
    vultr - Transferred: 719.438 MiB / 719.438 MiB, 100%, 5.948 MiB/s, ETA 0s

  • using this command
    rclone copy ./file.ext remote: -vv --progress --s3-upload-concurrency=16 --s3-chunk-size=64M
    wasabi - Transferred: 719.438 MiB / 719.438 MiB, 100%, 67.034 MiB/s, ETA 0s
    vultr - Transferred: 719.438 MiB / 719.438 MiB, 100%, 52.077 MiB/s, ETA 0s

  • https://www.vultr.com/docs/how-to-use-rclone-with-vultr-object-storage/#Troubleshooting

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.