Start up Slow, then Too many requests

rclone sync can use a lot less api calls per second, as it takes time to transfer the files, the default is just four at a time and an api call per chunk.

you are using a client id, shared with a unknown count of fellow rcloners.
you can try using --tpslimit to slow down the number of api calls per second.
or try --fast-list to reduce the number of api calls.

perhaps @Ole has a suggestion?

Hi Marcelloh,

Your rmdirs command seems correct if you are trying to recursively remove all empty directories under remote:stack.

I do see some OneDrive throttling, but that could be OK if your have many folders/files and/or have had a lot of OneDrive activity in the past 24 hours. You therefore may get some good info from this post/thread on OneDrive throttling:

If the above post doesn't help, then I need some additional information to better understand your situation:

What are the characteristics of your data in remote:stack? How many folders? How many empty folders? How many files? Total size? Anything special?

What are the characteristics of the past 24 hours activity on your OneDrive? (both rclone and native clients)

Now I'm trying this:

./Applications/RClone/rclone rmdirs remote:stack --dry-run --transfers=2 --checkers=2 -vv 
2021/07/02 10:54:00 DEBUG : Using config file from "/Users/marcelloh/.config/rclone/rclone.conf"
2021/07/02 10:54:00 DEBUG : rclone: Version "v1.55.1" starting with parameters ["./Applications/RClone/rclone" "rmdirs" "remote:stack" "--dry-run" "--transfers=2" "--checkers=2" "-vv"]
2021/07/02 10:54:00 DEBUG : Creating backend with remote "remote:stack"
2021/07/02 10:54:13 DEBUG : Too many requests. Trying again in 311 seconds.
2021/07/02 10:54:13 DEBUG : pacer: low level retry 1/10 (error activityLimitReached: throttledRequest: The request has been throttled)
2021/07/02 10:54:13 DEBUG : pacer: Rate limited, increasing sleep to 5m11s
2021/07/02 10:54:14 DEBUG : pacer: Reducing sleep to 3m53.25s
2021/07/02 10:54:14 DEBUG : pacer: Reducing sleep to 2m54.9375s

Idk how many empty folders, it's almost 300GB
My source:
296164 files
79241 folders
The last time I've accessed OneDrive was via Rclone yesterday.

I see, what did you learn by reading the thread I linked above?

That I can have extra params which I lowered to make sure, but without any success?

There is a lot more to be learned :grinning:
I suggest you read the entire thread again and pay attention to all details.

But that thread is about copying from OneDrive to GooglDrive, while I'm copying from localdrive to OneDrive.
What would be the slowest, most throttled command for that?

I now have this:

./Applications/RClone/rclone rmdirs remote:stack --dry-run --transfers=1 --checkers=1 -vv --progress --stats 10s 

Without success

Correct, but there is still a lot to be learned from that thread :grinning:
I suggest you read it and pay attention to all the details.

It must be a guessing game, well I'm more a person that likes to read solutions without having to solve a puzzle. Thanks for trying to help.

i offered several possible solutions, none of them helped?

keep in mind that each time you run your command with --dry-run, you are using a huge amount of api calls and nothing is getting removed.

I now try this one:

./Applications/RClone/rclone rmdirs remote:stack --transfers=1 --checkers=1 -vv --progress --stats 10s --tpslimit 1 --fast-list

I will let this run for a while, but for now, it's only showing me:
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Elapsed time: 5m11.9s
(so 5 minutes doing nothing at all)

I probably will after I;m back from my holiday. Have more time then.

Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Elapsed time:     20m41.9s

Back from holiday and made a client (Microsoft OneDrive)
I copied the client-id and the secret-id into an empty text document, edited the configuration and ran it again.
The first time it complains about an something being wrong (I think it was the ID).
Then when I do it again, it's just the same situation:

./Applications/RClone/rclone rmdirs remote:stack --transfers=1 --checkers=1 -vv --progress --stats 10s --tpslimit 1 --fast-list
2021/07/12 09:10:34 DEBUG : Using config file from "/Users/marcelloh/.config/rclone/rclone.conf"
2021/07/12 09:10:34 INFO  : Starting transaction limiter: max 1 transactions/s with burst 1
2021/07/12 09:10:34 DEBUG : rclone: Version "v1.55.1" starting with parameters ["./Applications/RClone/rclone" "rmdirs" "remote:stack" "--transfers=1" "--checkers=1" "-vv" "--progress" "--stats" "10s" "--tpslimit" "1" "--fast-list"]
2021/07/12 09:10:34 DEBUG : Creating backend with remote "remote:stack"
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Elapsed time:      4m52.1s

So I edited it again, setting both to empty again.
Then run and stop.
Then edited it again, setting the 2 id's, to see that one time message:

If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=_L0-d1TpFmiBi1qJM5D8Ng
Log in and authorize rclone for access
Waiting for code...
Got code
2021/07/12 09:46:15 Failed to configure token: failed to get token: oauth2: cannot fetch token: 401 Unauthorized
Response: {"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret is provided.\r\nTrace ID: 88bbda57-7836-45aa-84a4-ad7b70595200\r\nCorrelation ID: e0f4a7eb-c851-4f0c-88c3-bf1153786c87\r\nTimestamp: 2021-07-12 07:46:17Z","error_codes":[7000215],"timestamp":"2021-07-12 07:46:17Z","trace_id":"88bbda57-7836-45aa-84a4-ad7b70595200","correlation_id":"e0f4a7eb-c851-4f0c-88c3-bf1153786c87","error_uri":"https://login.microsoftonline.com/error?code=7000215"}

hi,

about the cliend id, make sure to use value, not Secret ID

In the instructions it shows "Copy and keep that secret for later use." and since the other one was about the "id" i'd figured this one was about the "id" as well.
I didn't copy that, so have to remove and retry.

No more errors when editing the config, but still:

2021/07/12 15:23:32 DEBUG : Creating backend with remote "remote:stack"
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Elapsed time:       2m2.0s

ok, so now we are passed the config issue.

what is the problem now?

Idk, i I think it doesn't do what it supposed to do.

2021-07-12 16:23:00 DEBUG : remote: Saved new token in config file
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Elapsed time:   1h57m12.0s

what do you expect it to do?

to save time with going back and forth, you need to post the exact commands and a debug logs.

I want to get rid of the empty directories,and since I've tried so many different "advices" I'm not sure if this is even the right one.
(How sweet it could be if this was somewhere to be found on a page.)

./Applications/RClone/rclone rmdirs remote:stack --transfers=1 --checkers=1 -vv --progress --stats 10s --tpslimit 1 --fast-list
2021/07/12 15:23:32 DEBUG : Using config file from "/Users/marcelloh/.config/rclone/rclone.conf"
2021/07/12 15:23:32 INFO  : Starting transaction limiter: max 1 transactions/s with burst 1
2021/07/12 15:23:32 DEBUG : rclone: Version "v1.55.1" starting with parameters ["./Applications/RClone/rclone" "rmdirs" "remote:stack" "--transfers=1" "--checkers=1" "-vv" "--progress" "--stats" "10s" "--tpslimit" "1" "--fast-list"]
2021/07/12 15:23:32 DEBUG : Creating backend with remote "remote:stack"
2021-07-12 16:22:59 DEBUG : remote: Loaded invalid token from config file - ignoring
2021-07-12 16:23:00 DEBUG : Saving config "token" = "{\"access_token\":\"eyJ0eXAiOiJKV1QiLCJub25jZSI6Ik9zbU5JZEpzTEFzX3FZTzlOUDVGa3VKTlpWajhTOFdmbVhyWkZfOFhsQjgiLCJhbGciOiJSUzI1NiIsIng1dCI6Im5PbzNaRHJPRFhFSzFqS1doWHNsSFJfS1hFZyIsImtpZCI6Im5PbzNaRHJPRFhFSzFqS1doWHNsSFJfS1hFZyJ9.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTAwMDAtYzAwMC0wMDAwMDAwMDAwMDAiLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC9hNWU5MzFkNi02OTEwLTQ1ZDQtODAzYy01MzY5MzQxZTVkMWYvIiwiaWF0IjoxNjI2MDk5NDgwLCJuYmYiOjE2MjYwOTk0ODAsImV4cCI6MTYyNjEwMzM4MCwiYWNjdCI6MCwiYWNyIjoiMSIsImFjcnMiOlsidXJuOnVzZXI6cmVnaXN0ZXJzZWN1cml0eWluZm8iLCJ1cm46bWljcm9zb2Z0OnJlcTEiLCJ1cm46bWljcm9zb2Z0OnJlcTIiLCJ1cm46bWljcm9zb2Z0OnJlcTMiLCJjMSIsImMyIiwiYzMiLCJjNCIsImM1IiwiYzYiLCJjNyIsImM4IiwiYzkiLCJjMTAiLCJjMTEiLCJjMTIiLCJjMTMiLCJjMTQiLCJjMTUiLCJjMTYiLCJjMTciLCJjMTgiLCJjMTkiLCJjMjAiLCJjMjEiLCJjMjIiLCJjMjMiLCJjMjQiLCJjMjUiXSwiYWlvIjoiRTJaZ1lLZ0t1T3lqbUZ0OC9PdWNseTlaeld5K0cycTZDWitWK0NER0k4UHB4dkphd2hzQSIsImFtciI6WyJwd2QiXSwiYXBwX2Rpc3BsYXluYW1lIjoicmNsb25lIiwiYXBwaWQiOiI4N2Q0OWM5MS04NTE3LTQwNTAtYTJiMy03NTA0ZjM0MDk5Y2EiLCJhcHBpZGFjciI6IjEiLCJpZHR5cCI6InVzZXIiLCJpcGFkZHIiOiI4OS4yMjAuMTUwLjk0IiwibmFtZSI6Ik1hcmNlbGxvIEgiLCJvaWQiOiJjMWMxNzgxZC1iMGUyLTQ3ODAtYjYzMS1hNjA1ODUzMGZkZTUiLCJwbGF0ZiI6IjUiLCJwdWlkIjoiMTAwMzIwMDBEN0VGMTBCQyIsInJoIjoiMC5BVW9BMWpIcHBSQnAxRVdBUEZOcE5CNWRINUdjMUljWGhWQkFvck4xQlBOQW1jcEtBTHcuIiwic2NwIjoiRmlsZXMuUmVhZCBGaWxlcy5SZWFkLkFsbCBGaWxlcy5SZWFkV3JpdGUgRmlsZXMuUmVhZFdyaXRlLkFsbCBTaXRlcy5SZWFkLkFsbCBwcm9maWxlIG9wZW5pZCBlbWFpbCIsInNpZ25pbl9zdGF0ZSI6WyJrbXNpIl0sInN1YiI6IjNzN3Zmbjl6Uk5iM0JiS3pGYmdJZklwTWZuNWlFdWpMOVB0eTNmeGpsbEkiLCJ0ZW5hbnRfcmVnaW9uX3Njb3BlIjoiQVMiLCJ0aWQiOiJhNWU5MzFkNi02OTEwLTQ1ZDQtODAzYy01MzY5MzQxZTVkMWYiLCJ1bmlxdWVfbmFtZSI6IjE0MzY0MjlAc2RueWR4Lm9ubWljcm9zb2Z0LmNvbSIsInVwbiI6IjE0MzY0MjlAc2RueWR4Lm9ubWljcm9zb2Z0LmNvbSIsInV0aSI6IlhOaGhtNlRBb0VxZ3FVa3VzcHhpQXciLCJ2ZXIiOiIxLjAiLCJ3aWRzIjpbImI3OWZiZjRkLTNlZjktNDY4OS04MTQzLTc2YjE5NGU4NTUwOSJdLCJ4bXNfc3QiOnsic3ViIjoic2FYUnNxSmlkcG9mRW5HengzVHlTblNSZlRmTGVhc1J3NlhwZTFFWGFoVSJ9LCJ4bXNfdGNkdCI6MTU1MDk5NTM2MH0.NuDo4bLj48DPsb38sJKTcTsJKLoR4-YXF5_3BsM9lCV_T9vbC9IDgE99VNHYrLz_ZXL-FR7nFLu0F1Re4Y5fVvIMZdbAg7hDw5cYhWtVoNCyK3oIA3HfiZ-XDdFY1Mkv6KAH_CnTdNiCMJtptyvsK2I06AyVAzuVrdKGMqYwt_bQQ3Y2XIFQRuDsZaShk6xctvjVdO9dQFF0ETl3DObcp7Oc9xozbue60XeBYOuuTbi9Cv5WM91D1l6sCbtxj-putxSSNNUmYKJ_I-q6bVWUTdR6yzp_pMCy85XlAQPO7VAmUylB8ZIhAW6IYmZAydt09H_bmLQ6Ao8aXUePbhIBSQ\",\"token_type\":\"Bearer\",\"refresh_token\":\"0.AUoA1jHppRBp1EWAPFNpNB5dH5Gc1IcXhVBAorN1BPNAmcpKALw.AgABAAAAAAD--DLA3VO7QrddgJg7WevrAgDs_wQA9P8QRPKxLXGSaGv7GZCDA5iLC8fFiYZ5dQa6PR-R-DyOf2aBOKk5nOVD4YVYJMkrUpzxKdiA8mfQo50Ij4gfH7YjYkVtB_RMbtasHRFubqHXHYB-m3y9wbJX77qtnLuXO2ez4Pz2dtuIp77MHrQo99WzR-fFS2Ui1n0NVb8fVcf6pvaSED20qsi7BuXUL9M-VUm8J1fEtGi5PwOW7MjOZvRMcTCiXmXx3QnRzOb8ZGnIOvL504iZ5DjoEJcDwUsRlh60m4vMhqJdC1QKrk2em8rudK45hCveJNtt3pAtKLK245q75-B_3B9ESS1LMbqIv3ugKFIkz9WYRtTYlPfAY6Gz5YoNomGPD8Kn7TGZED9KoH3UmX-Smgp4-sGqWFazKCMpjz9NtaTRwLCuu3YwNxxqZIzSUJYNUiGs6Mxhz-7EpUgTUeIZL9TEOJYeqvwPrgfn5ci86klvlDbbvaL4WGM-WqXpPVqRjJU93sEXDY68pKk8mN22QP21hMbIQRx_-5IXm-07515r8_T2CKC_BfLSL5S0T9My6ZwV7LMzr81UpM07zr61evWGJRF9Iy0WTVjJ-8iNer6x_ETb6zOJpeBlG4FxC1rmlMInlxCIcxWlVBSsi0wwcmf_RyKscS9BtTWT1AAFnjMUBhVLxzh3kqM8OPD_2_-JVoT98EuDJLQ-2Bl6mHXl4TV-PESn_WuSo0bEEvvkoPSByKd-tnnLFwQkqIBBZq6b4RBUpD2qn1t_iDpVPqC92dHxZJTSuXsWxjuxlJadU34l8I42M5bHjw8yo2J1ENQW8dnz6UqLfs8m7L1pSbWHMd4MKtpn3Xlsa5HGI4CDy71GweV4IOvTrjhJVq9RbzmJXoSPHHx0oHv6c-gwJklCwhRjARfi3PV4jvDbu_p3nYP1Xj-pmojFLooX6jsYKmkTwS7FWgwKOrV__SAE8IcBKiy6MgBATmC63O23fOx7H4ah61Ax4YYv\",\"expiry\":\"2021-07-12T17:22:59.633722+02:00\"}" in section "remote" of the config file
2021-07-12 16:23:00 DEBUG : remote: Saved new token in config file
2021-07-12 17:22:50 DEBUG : remote: Loaded invalid token from config file - ignoring
2021-07-12 17:22:51 DEBUG : Saving config "token" = "{\"access_token\":\"eyJ0eXAiOiJKV1QiLCJub25jZSI6ImM0V2pkdFpJMjBKRXdzVVBsbnhxTGUxTmd1SkVaQmllNmFnVWlxb2lxaWsiLCJhbGciOiJSUzI1NiIsIng1dCI6Im5PbzNaRHJPRFhFSzFqS1doWHNsSFJfS1hFZyIsImtpZCI6Im5PbzNaRHJPRFhFSzFqS1doWHNsSFJfS1hFZyJ9.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTAwMDAtYzAwMC0wMDAwMDAwMDAwMDAiLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC9hNWU5MzFkNi02OTEwLTQ1ZDQtODAzYy01MzY5MzQxZTVkMWYvIiwiaWF0IjoxNjI2MTAzMDcwLCJuYmYiOjE2MjYxMDMwNzAsImV4cCI6MTYyNjEwNjk3MCwiYWNjdCI6MCwiYWNyIjoiMSIsImFjcnMiOlsidXJuOnVzZXI6cmVnaXN0ZXJzZWN1cml0eWluZm8iLCJ1cm46bWljcm9zb2Z0OnJlcTEiLCJ1cm46bWljcm9zb2Z0OnJlcTIiLCJ1cm46bWljcm9zb2Z0OnJlcTMiLCJjMSIsImMyIiwiYzMiLCJjNCIsImM1IiwiYzYiLCJjNyIsImM4IiwiYzkiLCJjMTAiLCJjMTEiLCJjMTIiLCJjMTMiLCJjMTQiLCJjMTUiLCJjMTYiLCJjMTciLCJjMTgiLCJjMTkiLCJjMjAiLCJjMjEiLCJjMjIiLCJjMjMiLCJjMjQiLCJjMjUiXSwiYWlvIjoiQVNRQTIvOFRBQUFBVHl4L1lwalFWQXlLZ2xTazc3RjlBdm9Nb1R0anFOR3o1OUkvb2RpVmtGST0iLCJhbXIiOlsicHdkIl0sImFwcF9kaXNwbGF5bmFtZSI6InJjbG9uZSIsImFwcGlkIjoiODdkNDljOTEtODUxNy00MDUwLWEyYjMtNzUwNGYzNDA5OWNhIiwiYXBwaWRhY3IiOiIxIiwiaWR0eXAiOiJ1c2VyIiwiaXBhZGRyIjoiODkuMjIwLjE1MC45NCIsIm5hbWUiOiJNYXJjZWxsbyBIIiwib2lkIjoiYzFjMTc4MWQtYjBlMi00NzgwLWI2MzEtYTYwNTg1MzBmZGU1IiwicGxhdGYiOiI1IiwicHVpZCI6IjEwMDMyMDAwRDdFRjEwQkMiLCJyaCI6IjAuQVVvQTFqSHBwUkJwMUVXQVBGTnBOQjVkSDVHYzFJY1hoVkJBb3JOMUJQTkFtY3BLQUx3LiIsInNjcCI6IkZpbGVzLlJlYWQgRmlsZXMuUmVhZC5BbGwgRmlsZXMuUmVhZFdyaXRlIEZpbGVzLlJlYWRXcml0ZS5BbGwgU2l0ZXMuUmVhZC5BbGwgcHJvZmlsZSBvcGVuaWQgZW1haWwiLCJzaWduaW5fc3RhdGUiOlsia21zaSJdLCJzdWIiOiIzczd2Zm45elJOYjNCYkt6RmJnSWZJcE1mbjVpRXVqTDlQdHkzZnhqbGxJIiwidGVuYW50X3JlZ2lvbl9zY29wZSI6IkFTIiwidGlkIjoiYTVlOTMxZDYtNjkxMC00NWQ0LTgwM2MtNTM2OTM0MWU1ZDFmIiwidW5pcXVlX25hbWUiOiIxNDM2NDI5QHNkbnlkeC5vbm1pY3Jvc29mdC5jb20iLCJ1cG4iOiIxNDM2NDI5QHNkbnlkeC5vbm1pY3Jvc29mdC5jb20iLCJ1dGkiOiJJMGI1TVRsenNVR2syNTZhMjV4MUF3IiwidmVyIjoiMS4wIiwid2lkcyI6WyJiNzlmYmY0ZC0zZWY5LTQ2ODktODE0My03NmIxOTRlODU1MDkiXSwieG1zX3N0Ijp7InN1YiI6InNhWFJzcUppZHBvZkVuR3p4M1R5U25TUmZUZkxlYXNSdzZYcGUxRVhhaFUifSwieG1zX3RjZHQiOjE1NTA5OTUzNjB9.FaHnYgdeKqEPVFI1cYJLx89lBP5oDY43Bvaa2HklxJScXL1dgSdGYwKg4ABCEIDFa_S8wDEV6MGPZuoapIJjHmU0ouvyrFPzLeq0XK5H_LVc3aq0XtykMeyNjUuDuDU182tKtlWFv1AgniDx7dMgs5glAUnOjcSGj3kP02tDsDYMHoc6y6OUs5E9optkwnpuYuogsFirU-xOymE1yO3gk__lI1kdOTjQw8DWnlSa2j4rpIwFNFPtm0YyftvDjLy00TlBZ3M6DXcKOj-hgEI1zH8RRydRziPwrS7q5llVCtp6QQ4iV9Fqq3am-fL3C1adZ_YaqK8rq3GoGgez5h3acg\",\"token_type\":\"Bearer\",\"refresh_token\":\"0.AUoA1jHppRBp1EWAPFNpNB5dH5Gc1IcXhVBAorN1BPNAmcpKALw.AgABAAAAAAD--DLA3VO7QrddgJg7WevrAgDs_wQA9P8f-PjzqXD0K_mKT3x4CETWYaiAujK1vcudYXKFZEaMYpguS2KaKAP-pDSLkATtGjyuOSXTqgDd8zuj-lnWNCKOt0-xYpjzQB7zK--cNUlosn182ZRna7EtkrJ_p74lwsGxyx63ClHsKb61DG5Q1D6-80OW3rBNnu0_x1PkCYfJJVmTw1w_g3ZBuR36r4v5dQ9943zf_VBXEjL-S7r5OAzI_oweNql477GMuGywk7D2tkG3FtXD0716sL-sJRb7itB1bPAbx3Jov4eui8ZZHtqjHGs5oCL9IRWgbvfKYhYMU5XBYusEeGGbrgWFf-O4_FaAHjw8Os3u2ryqDeVSc7BycJRHPkkRcqXr5-AgxOqKtZW_5JiqpPxaDZxGuRhB_jlGj767QczfG6SN4iW8H93eQ1lOweNM2C7wqLMhT8H9e62BnPW5LrkfnAJFvGsB7Kw9EO99e-pmcRS-pxtkEohTtgLT8810730nDSY_0xA6XGhxGVbwCunLDaXONVapH221PH9zpZiBb5tIV_w5HkLb0oG0NqbtPN2ePLHJ0jSiprRc0wRY2L4FQ6u17tZgQLhMtbgEpG5zfnsZUDQLzr2swxm8QUUdri0I0PsKMuVvcUNpp8-0i0tou_2Tajba7lxoUhRjn_xAMQDolH4pyco7770a7FSsDL6sJ6NWdxUy1puDU9p3QF1FypgpmeTZctiu5FD7j2wDa6ePSFErLQ4LZYNPoabVrR473tiS6EVizkJsmWJViuYvKT4UFC_ePkiF1O4Ygz5V7X8wh6Op6vyTeN-Gmt_fDYT7YRfCdVRwwJCuvW9maiZqnrPXWtq4iPlaeRvMfqoBnkf4LmcOB9EIMFpCL9XhkHsbORlPy5ADlX6D46_kG484EfI-dQrp1ZF_hsWfr9K30tPmO3SvVJowVv5B8xJpWkMPzFg55s8RybkOGVYyFFOAZf95hIduadG7BnM4ro8dagI2lF6e\",\"expiry\":\"2021-07-12T18:22:50.221587+02:00\"}" in section "remote" of the config file
2021-07-12 17:22:51 DEBUG : remote: Saved new token in config file
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Elapsed time:   2h26m12.0s

rmdirs reqiures that folders to be empty.

if the folders are not empty, use rclone purge