Some updates!
So the --vfs-cache-mode full flag did take quite a while since the file was so large, but once it was done, the same issues occurred again (periodic buffering). I thought this was extremely odd since this was all local.
I was using the modem/router combo that my ISP gave me since it was required and was using an additional router as repeat/extension. I had two of these routers from a previous set up, so I changed the modem/router from my ISP to passthrough so it would only use the two routers that I owned.
After this change, I was able to play content without any issues! Using my previous command, I was able to get large files running in about 7 seconds. My next goal is to make the start up time a little shorter. I was reading through this thread: Plex playing, buffering, stoppage - rclone mount issues, and I noticed @VBB was using the following commands for quicker access:
title Rclone Prime
D:\Programs\Rclone\rclone rc vfs/refresh recursive=true --fast-list -v
pause
I'm going to test this out, but just curious if you had any experience with it?