Using --files-from with Drive hammers the API

I pasted some representative logs in another thread.

The number of files varies considerably. Sometimes it is hundreds or thousands of tiny files with some big ones mixed in, other times it is fewer than a dozen, mostly large files. It seems like the issue scales with the number of files because I only noticed it when I found a background job that had hung since November (!), which prompted me to update rclone and try to clear out the backlog, which had grown to thousands of files.

What it appears is happening is that rclone is taking some steps before it even gets to checking files that causes a huge spike in API calls and then it ends up spending most of its time throttling API calls. I have left it running for upwards of 9 hours with no change. The log looks pretty much exactly like what I pasted in the other thread, over and over and over, though sometimes a few files might actually transfer before it freezes again.

It feels to me like --files-from is causing it to loop over each file and perform some kind of API heavy operation that does not happen otherwise.

Adding --tpslimit has no effect, but using --no-traverse results in the checking / transfers starting, but with extremely low (single kb/s) upload speeds.

Switching between --files-from and just pointing it right at the directory is like magic; the latter fires up and starts checking files immediately and begins uploading right away.