welcome to the forum,
that should not be the case,
at most, there should be a 1 minute delay, based on --dir-cache-time 1m
you need to trigger rclone to re-scan the directory, but navigating into it.
to see exactly what rclone is doing, use -vv for debug output.
--poll-interval 10
does nothting, as s3 remotes do not support polling.
--vfs-cache-mode=full --vfs-cache-mode=off
need to pick one, not both.
and can check out my summary of the two rclone vfs caches.
https://forum.rclone.org/t/status-about-using-rclone-for-music-storage-playback-in-2021-access-times-improved/27648/34