I'm trying to copy tens of thousands of assets from Google Drive to Google Photos. I've tried a whole bunch of options, including transfers from 1 to 20 and tpslimit as low as 1, but no matter what I try I keep getting lots of
error Quota exceeded for quota 'concurrent write request' of service 'photoslibrary.googleapis.com'. (429 RESOURCE_EXHAUSTED)
I am watching how often I get Committed sync batch length __ starting with: messages in the log, as that seems to indicate "actual" successful uploads, and it does a few dozen in a given hour, then nothing at all for 2-4 hours.
Hour batches assets
2024/12/05 14 138 786
2024/12/05 15 86 493
2024/12/05 16 0 0
2024/12/05 17 0 0
2024/12/05 18 13 79
2024/12/05 19 8 13
2024/12/05 20 2 3
2024/12/05 21 15 96
2024/12/05 22 0 0
2024/12/05 23 15 83
2024/12/06 00 0 0
2024/12/06 01 12 60
2024/12/06 02 13 63
2024/12/06 03 0 0
2024/12/06 04 5 18
2024/12/06 05 156 903
2024/12/06 06 0 0
2024/12/06 07 0 0
2024/12/06 08 0 0
2024/12/06 09 2 28
2024/12/06 10 0 0
2024/12/06 11 1 1
Some hours have hundreds of successful uploads, other have zero.
Very frustrating.
My current command is
rclone copy -Pvv \
--transfers 16 \
--tpslimit 1 \
--drive-chunk-size 128M \
--exclude "*.{xml,bin,json,thm,DS_Store}*" --ignore-case \
--log-file=rclone-photos-"$(date +%Y%m%d_%H%M%S)".log \
gdrive:___ gphotos:album/rclone"
I tried the option gphotos-batch-mode async but that crashed as I reported here: `--gphotos-batch-mode async` error · Issue #8233 · rclone/rclone · GitHub
Can anything be done to make this more efficient?
$ rclone version
rclone v1.68.2
- os/version: ubuntu 24.04 (64 bit)
- os/kernel: 6.8.0-1015-oracle (aarch64)
- os/type: linux
- os/arch: arm64 (ARMv8 compatible)
- go/version: go1.23.3
- go/linking: static
- go/tags: none