I'd like to disable multipart uploading for a S3 remote, but I did not find such a flag. It is also possible that my plan does not make sense.
I want to sync a set of files to a S3 Deep Glacier crypt remote. I'm aware that verification might not work if both crypt and multipart-uploading are in effect.
My plan is:
rclone copy my files to the S3 crypt remote, while disabling multipart uploading
- Check which files failed to upload,
rclone copy these files to a local crypt remote
rclone copy from the local crypt remote to the S3 remote (without crypt)
Other plans that I have considered:
rclone cryptcheck ==> This would be too slow for me, and it might not work for deep glacier.
Copy all files to local crypt first ==> I think it is too slow
Only directly upload small files ==> I suppose a file smaller than 3GB won't get bigger than 5GB after the encryption, but I'm not sure about the optimal cutoff.
i use aws s3 deep glacier, have many dozens of TiB stored there.
sorry, not possible to disable multi-part uploads using s3.
once a file is over 5GiB, have to use multipart.
i could be wrong but the best is
for non-technical and the cost of api calls, i am not able rclone crypt with aws s3 deep glacier.
i only upload local files that are already crypted; veeam backup files and .7z
so i can use
Hi, thanks for you reply!
To clarify, I meant "disable multipart uploading and show an error if the encrypted file is too big".
there would not be an error, as rclone will upload any file you asked it to.
The error should be "the file is too big to upload without multipart uploading"
as far as i know, no such error.
if you want to limit files to sizes below 5GiB.
This is one of the options I listed above.
The issue is, if a file is just a little bit smaller than 5GB, the encrypted file may be larger than 5G, and thus trigger multipart uploading.
I could probably use 4GB as the threshold, but I'm here asking whether there are better solutions.