a file will only be downloaded, if some local apps downloads it and there should not be a delay.
the issus here about keeping the rclone vfs dir cache up to date.
there are two caches that rclone can use, read about it
https://forum.rclone.org/t/status-about-using-rclone-for-music-storage-playback-in-2021-access-times-improved/27648/34
--dir-cache-time Duration Time to cache directory entries for (default 5m0s)
you can lower that value, but as a result, willl increase the number of api calls to aws.