OneDrive: rclone getting throttled and pacer sleep time skyrockets to >5m

[Ole (Ole Frost)]

Hi Francesco,

@Francesco_Pretto

rclone should either behave so it will get throttled less or the pacer should not select these (nonsense?) huge sleep times.

The sleep times you see are due to rclone respecting the Retry-After time returned in the HTTP 429 (Too many requests) responses from OneDrive. There is no way to avoid them for sure, they depend on many parameters including the current load on the OneDrive servers hosting your data.

You can see it being received and handled in this line of your log:

2021-10-02 18:56:48 DEBUG : Too many requests. Trying again in 322 seconds.

You can however find some tips and experiences to reduce the throttling in this thread:

Rclone v1.55.1 throttling and worse on OneDrive copy to GD

I just ran my sync from OneDrive to GoogleDrive and it had the usual speed around 12 Mbyte/s. I use my own client ID on both remotes. The only tuning parameter is --checkers=16, the rest are defaults (that is no --tpslimit, --transfers=4 etc.) I have found that it is best to stay just under the OneDrive throttle limit all the time, and that corresponds to the rclone default settings in most situations. This is especially important with respect to transfers. OneDrive Personal will only do 4 tran…

And read more about OneDrive throttling in this article:

Avoid getting throttled or blocked in SharePoint Online

On a side note to illustrate the sporadic nature of the OneDrive throttling: I am looking at ways to improve the pacer and my current problem is that I am currently unable to provoke the HTTP 429 responses, despite very heavy load (disrespecting all of the recommendations in the above thread).