What I was wondering is if this was an interaction between Rclone mount and SMB or whether the mount itself was very slow. It sounds like it is probably the interaction which is the problem.
SMB has quite a few options read size / write size etc it might be worth having a go at tweaking some of those.