You're right that it seems config-related. Maybe more config than I expected (not just the one in rclone/config file). In the past few days I did change the following mount parameters (due to that other issue):
--transfers
--buffer-size
--b2-upload-concurrency
--multi-thread-streams
And even though I haven't touched any --vfs-* parameters, this seems to suggest that any filesystem-related configs could affect it (I haven't gotten to the bottom of it yet, just making assumptions for now).