I got the 404 error for 90 files instead of 92. On a 2nd run, 90 errors again.
At this point it seems clear that the issue is on the Google side which is good for rclone but not so much for me 
My only option seems to be retrying until I get all those objects, so I just grepped the paths that are returning 404 and piped them to a file. I'm looping through the file trying to rclone copy each object, hopefully this won't trigger any rate limits... let's see how that goes.