I'm on windows so neither of those flags do anything unfortunately. But definitely a good tip for others so thank you.
I'm almost fully certain that the files are not being downloaded, at least not their full contents. I monitored the network activity of rclone during a backup and it was around 18k bytes sent and received per second.
You mentioned cache dir data. Interestingly, the VFS cache does not get populated with folders or files basically at all. After a test restic backup of 10242 files (4.568GB), my rclone_cache folder is 503MB in Size, 4 files, 12 folders. Weirdly, the Size on disk is 11.3MB (I have CompactOS enabled). As a potential explanation, I found this comment which states that dir structure and file metadata is RAM only. There is no persistent metadata cache, evidenced by a 4 year old feature request for it on github.
I created a debug level log file for rclone serve, rclone mount, and restic backup. I'm not sure what to include, as the 3 log files combined were 1.5GB, but I can copy-paste certain sections if you have any suggestions. Here is a snippet of the rclone mount log during a restic backup.
I also found this comment by @ncw, and apparently mount does not issue S3 HEAD requests in parallel when listing files. If I had to guess this seems to be the crux of the issue.
I tested --checksum as the documentation recommends it for S3->S3 transfers, although I'm not sure if it works for mount. Also toyed around with --read-only, --no-update-dir-modtime, --no-update-modtime, --use-server-modtime. None of them made a difference.
All in all, I am fairly confident that the rclone mount speed is extremely limited and is based on cloud provider. In my testing 2 files per second was the lowest, and 20 files per second was the highest. So for example if I try to delete 200 .md files in the mounted S3 remote, it takes ~10 seconds. Deleting 2000 .md files -> 100 seconds. Thus, any file operation such as restic backup will be bottlenecked by this limitation.
Please let me know if I'm missing something, but based on this, creating restic snapshots of rclone remotes is just not feasible for any non-negligible amount of files. I'll have to rethink my backup strategy 