With Backblaze having now joined Amazon in supporting object-lock (and internal demand here) it would be beneficial if rclone could support object-locking for any S3-compatible target.
Resources:
With Backblaze having now joined Amazon in supporting object-lock (and internal demand here) it would be beneficial if rclone could support object-locking for any S3-compatible target.
Resources:
Looks interesting...
Want to have a go at sketching out how this might work with rclone? Which additional parameters rclone would need to have?
Hi, thanks for getting back to me.
In my imagining this would need work in the following areas as a first pass - I'm attempting to track calls with Amazon's API reference for consistency:
Bucket Creation (object lock must be set when the bucket is created for objects within it to be lockable):
--s3-bucket-object-lock-enabled
File upload operations:
--s3-object-lock-mode [GOVERNANCE|COMPLIANCE]
--s3-object-lock-retain-until-date [date]
--s3-object-lock-legal-hold [ON|OFF] (default OFF)
Delete operations:
--s3-bypass-governance-retention
Enumeration:
--s3-get-object-lock-configuration
--s3-get-object-retention
--s3-get-object-legal-hold
Error handling:
What do we return when a delete operation was run against a locked object?
API reference: https://docs.aws.amazon.com/AmazonS3/latest/API/API_Operations.html
Also I should mention that as far as I know this only applies to S3 implementations with Amazon and Backblaze, so it makes sense to only allow those flags to work with AWS and Other storage types maybe?
Perhaps this could be implied if any of the other object lock
modes are specified
I think those seem good. I think --s3-object-lock-mode
is the master setting. You'll need to supply that to get it to work at all. I think that --s3-object-lock-retain-until-date
might be better as --s3-object-lock-retention-period
so users can specify 1d
1y
etc from now. You can also put a date in here too 2021-01-01
.
I think this is optional isn't it?
Would people upload objects with a legal hold? I guess they might. However I was thinking an additional backend command to add/remove a legal hold might be appropriate rclone backend legal-hold add s3:bucket/path
OK
Rclone doesn't have a great way of surfacing these at the moment.
Maybe an additional rclone backend list-object-lock
command or similar might be appropriate.
S3 will return an error, I suggest we just return that.
That is easy enough to arrange in the configurator.
Thanks, this seems really promising. The only point I want to emphasize from the above is that object locking will work if and only if object-lock-enabled is set at the time of bucket creation, and only for objects within that bucket. For that reason I believe it's necessary to add that flag to 'rclone mkdir' - and if I'm wrong about this please accept my apology.
I think this is very useful. Would you mind writing it up as a GitHub issue? If you could link it here and vice versa that would be helpful. Thanks
Hi, sorry for the delay. I've created the GitHub issue per your request:
Thanks for doing that.
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.