Out of our 1GB terabyte nvme drive, recently we ran into a problem we're almost 800 GB of the files were being used by our screenshots, since the storage space was going to go beyond our current capacity; we decided we will move it to the cloud.
We had two options, 1) S3 or 2) Wasabi. We choose Wasabi due to ofcourse much lower pricing point.
Currently we are running into an interesting problem, whenever the screenshots are uploaded from our system, write seems to be super fast. Almost instantaneously. Which we are really happy about. However when our system tries to read the file back, it seems to be taking more than a minute each time.
I'm hoping someone can help us troubleshoot this a little bit more maybe they are using a wrong Mount command, but I'm not 100% sure.
Other options that we are exploring is Oracle object storage or perhaps backBlaze.
rclone mount seems to load files really slow.
rclone mount --allow-other --buffer-size 10G --fast-list --vfs-cache-mode writes --vfs-write-back 1s wasabi:bucket-name screenshots/ --daemon
[wasabi] type = s3 provider = Wasabi access_key_id = secret_access_key = region = us-east-1 endpoint = s3.wasabisys.com acl = public-read chunk_size = 50Mi disable_checksum = true upload_concurrency = 20
Not collecting logs atm.