I think I figured it out - sharing folders seems to take a long time, because google drive needs to recursively add access to each file. Piping rclone ls to wc -l after a few minutes started showing more and more files.
This issue took me completely by surprise and took a very long time to solve (I hope that's solved), because the folder I shared prior had thousands of files and while it seemed like it finished, in fact it was still being processed and the next folder that I tried to share despite being very small had to wait for the previous one. Bad luck I guess! But honestly, I will do everything I can to move away from cloud providers and set up a selfhosted file server accessed simply via ssh. This was a total nonsense.