Hmm there is another thread on the forum with a similar issue (may not be related though!)
What happens if you try rclone check
from the source to the dest?
Can you try rclone ls
the remote a few times and see if it gives the same results every time? It is as though there are files missing from the directory listing.