In November, 2018, Yandex team started test operation of their own Yandex.Cloud soultion. I had a chance to take part in Yandex Object Storage tests to check general compatibility its S3-like API with RClone v1.45.
No errors were found, and RClone - Yandex.Cloud pair works just fine.
Below is a couple of observations I’d like to share with community.
1. Storage Classes
There two storage classes in Yandex Object Storage, cold and standard (ref Concepts - Storage class). My choice for cloud backups is a cold storage because it’s less expensive than standard one.
Yandex cold storage class corresponds to Storage Classes for Infrequently Accessed Objects in Amazon Simple Storage Service.
RClone also has an appropriate option to define S3 storage class. But what I’ve noticed RClone config sequence doesn’t support entering/editing storage class for s3 - other remotes.
In order to set cold storage class for all objects in the bucket (remote) you may manually
- put following line in rclone.conf appropriate section
storage_class = STANDARD_IA
- or add command line option for each RClone call
--s3-storage-class STANDARD_IA
2. Multipart Uploads
Yandex Object Storage supports S3 multipart uploads. And while testing uploads to Storage I could see it clearly in case of sudden connection break. Hundreds of pending chunks were waiting for upload to complete or cancel.
I didn’t find RClone command to handle incomplete multipart uploads stuck due to connection failure. Similair case was reported by @peixotorms in his post Multipart Uploads to Digital Ocean Spaces.
To clear upload queue manually
- install AWS CLI (ref MS Windows example),
- issue list command (ref list-multupart-uploads)
aws s3api list-multipart-uploads --endpoint-url=https://storage.yandexcloud.net --bucket <name>
- save list of pending uploads
{ "Uploads": [ { "UploadId": "<number>", "Key": "<path>", "Initiated": "<time>", "StorageClass": "type" }, { ... } ] }
- issue abort command (ref abort-multipart-upload)
aws s3api abort-multipart-upload --endpoint-url=https://storage.yandexcloud.net --bucket <name> --upload-id <number> --key "<path>"
- repeat abort command for all pending uploads in the list
Note: –endpoint-url option is mandatory for Yandex Object Storage.
I do not think above observations can be treated as RClone feature requests. It’s more a matter of usability when working with a particular cloud service provider.