So using rclone -v --size-only sync drive:mirror/[redacted] . --drive-shared-with-me --checkers 9 --tpslimit 9 things have improved:
Checks: 437754 / 437754, 100%
Elapsed time: 6h14m52.8s
Is this you recommended value? Maybe I should make that the default? It is 10 TPS at the moment I think.
I think that the bug shows its head most often with shared with me stuff so it may not be in this case.
Most people, with google drive will have the max requests/sec of 10 requests, so it would make sense to make it the default for people using google drive. I can't speak about other backends.
If someone has a higher quota, then they would be advanced enough to increase it above the default....
Specifically it's cited in the Google developer console as 1000 requests per 100 seconds. Which, while it obviously does work out as an average 10 requests per second, is not quite the same thing.
Sure you can burst if you want and risk getting limited, which in the end will be slower than just limiting to 10
The default pacer should issue one API call per 100mS which is 10/s with a burst of 100
defaultMinSleep = fs.Duration(100 * time.Millisecond)
defaultBurst = 100
Maybe it is the burst which is the problem.
I just got hit with the fast list bug too in my teamdrive, which I never had issues before.
Was running a sync like always, then i noticed it was deleting a lot of stuff... I ran with --dry-run and it would have deleted over 30k files!!
Then I used the flag to disable fast list, and it worked ok 
Oops!
This sounds like a previous issue. What version of rclone were you using? Maybe the previous issue has come back
I'm using 1.53.1. On another server with 1.54 I'm having the issue that rclone often hangs at 100% forever if not killed. It will delete/rename/copy/etc everything but never finishes, not sure if it's google drive or rclone.
Will try to check it with different rclone versions to see if it helps
Same hang w 1.54. Sometimes. Not always. Odd.
I thought it was just me having a bad week lol.
OK, so I found something weird in my tests.
rclone v1.52.2-192-g8d5bc7f2-beta
[test]
type = drive
client_id = id
client_secret = secret
token = token
root_folder_id = 0AArHY_XXXXX[1]
type = drive
client_id = xxxxxx
client_secret = xxxx
scope = drive
service_account_file = /opt/sa-json/81.json
team_drive = 0AArHY_XXXXXXX
rclone size --fast-list gives a NOTICE log saying it disabled ListR with my non service account but with the same folder but using a service account fast list works.
My theory right now is that this is related to service accounts vs standard accounts. Some permission bug maybe?
I need to go soon, but I'll do lots of testing later to see if I can pinpoint the issue
1 Like
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.