Hello,
I just updated to 1.41 but it is a lot slower to copy/download a single file from google Drive.
With older versions, it just started the copy (.rclone copy remote:filename.xyz ./) right away but now it seems to get the whole directory first. This takes a lot of time with +100K files.
Output with -vv also shows “xyz Excluded from sync (and deletion)” for every file.
Is there something new that I must add to prevent rclone from downloading the whole file list every time?
The behaviour in the old versions was just the same. However I have just fixed this in the latest beta - can you try that?
I only have 1.36 on my laptop but I’m pretty sure it is the same for 1.39.
Can it have something to do with longer file names?
With a short file name, the latest Beta seems to work the same as the older versions.
With a longer filename (144 chars), I see this difference:
rclone v1.36:
Transferred: 957.667 MBytes (83.823 MBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 11.4s
rclone v1.41-035-ge42edc8eβ
Transferred: 957.667 MBytes (12.414 MBytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 1m17.1s
It just takes forever before the actual transfer starts.
If that doesn’t make sense, I can do some proper testing when I’m back in the office.
The thing to do will be to run with -vv --dump responses and then you can see exactly what rclone is doing with google drive. If you can post the two logs then that would be interesting.
A big difference between v1.36 and v1.41 is that we switched to using the v3 API from the v2 API in v1.40.
Strange... hopefully the logs will shed some light!
Hey, I have similar effect:
on rclone v1.37 i get :
Transferred: 16.547 kBytes (171 Bytes/s)
Errors: 0
Checks: 8481
Transferred: 1
Elapsed time: 1m39s
on rclone v1.42 it’s :
Transferred: 0 Bytes (0 Bytes/s)
Errors: 0
Checks: 8481
Transferred: 0
Elapsed time: 8m45s
same config same state on the local and remote side. The system I am running is different as well,
but this might be another argument for an issue ; )
I am not uptodate with all the changes came into the codebase, but has there been any fundamental changes in the transfers/checks etc. ?
Can you assist in improving the speed to what it was before ?!
One of the things was moving to the v3 API from the v2 API. I have had other reports that the v3 API is slower.
One thing you could try is download the latest beta and use the --fast-list flag.
on 1.42 with --fast-list it’s
Transferred: 16.547 kBytes (37 Bytes/s)
Errors: 0
Checks: 8481
Transferred: 1
Elapsed time: 7m27.1s
small improvement …
I will check the beta later. If there is no way to get things improved I think I have to downgrade to the latest version, which uses the old API, sadly.
ok, next try v 1.39:
Transferred: 128 kBytes (1.548 kBytes/s)
Errors: 0
Checks: 8481
Transferred: 4
Elapsed time: 1m22.6s
and the following version 1.40:
Transferred: 0 Bytes (0 Bytes/s)
Errors: 0
Checks: 8481
Transferred: 0
Elapsed time: 12m3.7s
finally, as pointed by @ncw the latest beta:
Transferred: 0 / 0 Bytes, 0%, 0 Bytes/s, ETA -
Errors: 0
Checks: 8481 / 8481, 100%
Transferred: 0 / 0, 0%
Elapsed time: 13m20.1s
From my point of view, the migration from gdrive api V2 to V3 has some reasonable lost in performace. Mhhhh… not good. Any insights if there are additional tasks to be done or any overhead ?
Seems that I have to stay with the old V2 api version. Factor 7+ slower is too slow …
Did you try that with --fast-list?
on 1.42, yes. are there changes in the newest beta ?
Yes! 1.42 does nothing with --fast-list whereas the beta will do lots!
oh, well… then i going to try the option with the latest and greatest beta 
ah @ncw,
look:
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors: 0
Checks: 8483 / 8483, 100%
Transferred: 0 / 0, -
Elapsed time: 1m32.3s
you were right. there is a significant speed-up. will stay with the beta and looking forward to the next release.
thanks for your support and all your superior work for rcone.
cheers !