Those settings look pretty good and very similar to the ones I use to serve beta.rclone.org.
I think if you are using --vfs-cache-mode full then you should probably check out this thread and try the latest beta posted there which should work much better.