Rclone don't stop sync process [Google Photos]

Help and Support
1

What is the problem you are having with rclone?

Rclone don't stop when all files synced. The process don't stop even all files completely synced. In log I have the problem below:

2020/10/07 17:37:09 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks: 7437 / 7437, 100%
Elapsed time: 30m0.1s

What is your rclone version (output from rclone version)
rclone v1.52.0

Which OS you are using and how many bits (eg Windows 7, 64 bit)
Linux 3.2.40 #25426 SMP Wed Jul 8 03:14:12 CST 2020 armv7l GNU/Linux synology_armadaxp_ds414

**Which cloud storage system are you using? (eg Google Drive)
Google Photos

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone -v sync /gphotos: /volume1/gphotos/

The rclone config contents with secrets removed.

[gphotos]
type = google photos
token = {"access_token":"

A log from the command with the -vv flag

Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks:              7437 / 7437, 100%
Transferred:            6 / 6, 100%
Elapsed time:      13m0.1s

2020/10/07 23:13:51 DEBUG : pacer: low level retry 6/10 (error Quota exceeded for quota metric 'Read requests' and limit 'Read requests per minute per user' of service 'photoslibrary.googleapis.com' for consumer 'project_number:202264815644'. (429 RESOURCE_EXHAUSTED))
2020/10/07 23:13:51 DEBUG : pacer: Rate limited, increasing sleep to 16.220893899s
2020/10/07 23:14:07 DEBUG : pacer: low level retry 3/10 (error Quota exceeded for quota metric 'Read requests' and limit 'Read requests per minute per user' of service 'photoslibrary.googleapis.com' for consumer 'project_number:202264815644'. (429 RESOURCE_EXHAUSTED))
2020/10/07 23:14:07 DEBUG : pacer: Rate limited, increasing sleep to 16.462384848s
2020/10/07 23:14:24 DEBUG : pacer: low level retry 3/10 (error Quota exceeded for quota metric 'Read requests' and limit 'Read requests per minute per user' of service 'photoslibrary.googleapis.com' for consumer 'project_number:202264815644'. (429 RESOURCE_EXHAUSTED))
2020/10/07 23:14:24 DEBUG : pacer: Rate limited, increasing sleep to 16.633721009s
2

Do you have more of the log? That only shows that you get a slow down request from the API which is a normal thing with Photos as the API is really low.

3

Updated logs. I have the same error with quota. Do you mean that main problem that I'm running out of quota and I need to wait till it will be update next day?

The problem is that I see attempts to copy/sync even when all files have been transfered and process don't complete.

UPD: Looks like problem fixed. Google photos have current structure
media
-by-day
-by-month
-by-year

So many files duplicates in different folders, same file can be at month folder and same file at year folder. So we don't need dups and to have files organaized by year will be fine. The problem dissapeared when I used syntax like this:

rclone -v copy gphotos:media/by-year/ /volume1/gphotos/media/by-year/

So process finished with sucessfull message:

2020/10/07 23:57:51 INFO  : There was nothing to transfer
2020/10/07 23:57:51 INFO  :
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks:              2496 / 2496, 100%
Elapsed time:        16.5s

Will check a few tricks later with folder again and let you know.

4

I think so but I am not 100% sure with Photos as I thought they had a really low quota per day.

We can see if someone else that uses it a bit more chimes in.

1 Like
5

Yes quotas are usually the problem here. You can slow rclone down with --tpslimit 1 say which helps.