S3 Glacier + Cache?

#1

hello all, so started doing a ton of copies right into s3 glacier storage class, --size-only works fine since that’s in the metadata - question is, if put a cache in front of the s3 glacier bucket would that speed up the compares since this is a giant dataset with small deltas? if I am doing --size-only is the metadata all that’s effectively being stored in the cache?

thanks

#2

I definitely think think it would help a lot in that use case. Just remember as only one process can access the cache at a time.