I did some brief testing to see what settings would get optimal upload speeds with RClone, and the following worked quite well, getting me over 1600 MB/s (megabytes, not megabits) with peaks over 2 GB/s.
Aside from increasing the --s3-upload-concurrency, --transfers and the chunk size, a critical setting is the --s3-disable-checksum parameter, since the calculation of the checksum seems to be single threaded, which significantly limits throughput.
I didn't do enough testing to verify if the values for these parameters are truly optimal, but I hope this can be of help regardless. If you find better settings, feel free to share them! Also, The --ignore-checksum parameter might not affect performance that much. (edit: it seems that the --ignore-checksum flag does nothing for multipart uploads anyways)
Testing was done on a GCP n2d-standard-48 with tier 1 networking enabled, which gives you 25Gbit egress capabilities.
Correct me if I'm wrong, but the documentation seems to imply that the --s3-disable-checksum doesn't actually affect any checks on in transit data:. s3-disable-checksum docs
If I'm correctly interpreting the documentation, all that does is calculate the checksum of the file locally, and store it in the metadata.
In other words, I think that the following scenario is possible regardless of setting the --s3-disable-checksum option:
rclone calculates the checksum -> uploads the file -> corruption happens in upload -> corrupt upload completes successfully with the checksum stored in the custom metadata. So there's no actual integrity check being disabled.
That option does matter though if you're planning on doing an integrity check the next time you download the file, at which point you can indeed use that checksum in the metadata for comparison. So if that's important to you, you can leave the --s3-upload-concurrency flag out. Without that flag I still got about 700MB/s with these settings, which is still respectable.
As for your comment about the --ignore-checksum flag potentially affecting in transit corruption checks, I looked into it and it seems like that flag has no effect for multipart uploads because rclone already ignores the etag whenever it doesn't look like an md5sum, which is the case for multipart uploads to most S3 compatible providers. I think that means that the post upload checksum verification is skipped regardless.
tl;dr, in the end it is a choice based on use-case, what level of paranoia you want.
if i disable a safety feature, and the blank hits the fan, i get fired.
i see that you clearly understand the issue and potential outcomes.