Copy new album to Google Photos is too slow

I need to create new albums to Google Photos but I want to call a new instance of rclone for each album.
Before loading files to Google Photos, rclone seams to list all the files in cloud. Is it possible to skip the listing of remote files?
I'm using these commands:

  • rclone mkdir "google_photos:album/AlbumName"

  • rclone copy AlbumPath google_photos:album/AlbumName --verbose --include *.jpg --include *.mp4 --order-by name,ascending --transfers 1

    Thank you!

hello and welcome to the forum,

rclone has to compare the source and dest, to know which files need to be copied.

there are some flags that may help, depending on your use-case
https://rclone.org/docs/#no-check-dest
https://rclone.org/docs/#no-traverse

Hi,
thank you for your answer but nothing changed.
This is my log. In bold the slow events (20 and 60 seconds). Do you know if the delay is due to the Google API calls?

Thank you!

Loading ".\2018-04-23" in "google_photos:album/2018-04-23"
2021/02/09 10:02:26 DEBUG : rclone: Version "v1.52.2" starting with parameters ["rclone.exe" "mkdir" "google_photos:album/2018-04-23" "--no-check-dest" "--no-traverse" "-vv"]
2021/02/09 10:02:26 DEBUG : Using config file from "C:\Users\\.config\rclone\rclone.conf"
2021/02/09 10:02:26 DEBUG : Google Photos path "album/2018-04-23": Making directory
2021/02/09 10:02:26 DEBUG : Google Photos path "album/2018-04-23": Mkdir: dir=""
2021/02/09 10:02:26 DEBUG : google_photos: Loaded invalid token from config file - ignoring
2021/02/09 10:02:26 DEBUG : Keeping previous permissions for config file: -rw-rw-rw-
2021/02/09 10:02:26 DEBUG : google_photos: Saved new token in config file
2021/02/09 10:03:46 DEBUG : Google Photos path "album/2018-04-23": >Mkdir: err=
2021/02/09 10:03:46 DEBUG : 3 go routines active
2021/02/09 10:03:46 DEBUG : rclone: Version "v1.52.2" starting with parameters ["rclone.exe" "copy" ".\2018-04-23" "google_photos:album/2018-04-23" "-vv" "--include" ".jpg" "--include" ".mp4" "--order-by" "name,ascending" "--transfers" "1" "--no-check-dest" "--no-traverse"]
2021/02/09 10:03:46 DEBUG : Using config file from "C:\Users\\.config\rclone\rclone.conf"
2021/02/09 10:03:46 DEBUG : fs cache: renaming cache item ".\2018-04-23" to be canonical "//?/./2018-04-23"
2021/02/09 10:03:46 DEBUG : Google Photos path "album/2018-04-23": Waiting for checks to finish
2021/02/09 10:03:46 DEBUG : Google Photos path "album/2018-04-23": Waiting for transfers to finish
2021/02/09 10:03:46 DEBUG : Google Photos path "album/2018-04-23": Put: src=IMG_20180423_110118.jpg
2021/02/09 10:03:46 DEBUG : IMG_20180423_110118.jpg: Update: src=IMG_20180423_110118.jpg
2021/02/09 10:04:46 INFO :
Transferred: 0 / 1.010 GBytes, 0%, 0 Bytes/s, ETA -
Transferred: 0 / 214, 0%
Elapsed time: 59.8s
Transferring:
* IMG_20180423_110118.jpg: 0% /6.038M, 0/s, -
2021/02/09 10:05:02 DEBUG : IMG_20180423_110118.jpg: >Update: err=
2021/02/09 10:05:02 DEBUG : Google Photos path "album/2018-04-23": >Put:
2021/02/09 10:05:02 DEBUG : IMG_20180423_110118.jpg: Size:
2021/02/09 10:05:02 DEBUG : IMG_20180423_110118.jpg: >Size:
2021/02/09 10:05:02 INFO : IMG_20180423_110118.jpg: Copied (new)
2021/02/09 10:05:02 DEBUG : Google Photos path "album/2018-04-23": Put: src=IMG_20180423_110127.jpg
2021/02/09 10:05:02 DEBUG : IMG_20180423_110127.jpg: Update: src=IMG_20180423_110127.jpg
2021/02/09 10:05:09 DEBUG : IMG_20180423_110127.jpg: >Update: err=
2021/02/09 10:05:09 DEBUG : Google Photos path "album/2018-04-23": >Put:
2021/02/09 10:05:09 DEBUG : IMG_20180423_110127.jpg: Size:
2021/02/09 10:05:09 DEBUG : IMG_20180423_110127.jpg: >Size:
2021/02/09 10:05:09 INFO : IMG_20180423_110127.jpg: Copied (new)
2021/02/09 10:05:09 DEBUG : Google Photos path "album/2018-04-23": Put: src=IMG_20180423_110150.jpg
2021/02/09 10:05:09 DEBUG : IMG_20180423_110150.jpg: Update: src=IMG_20180423_110150.jpg

Rclone is listing all the albums. Do you have a huge number of albums?

The only way in the google photos API to look up an album by name is to look them all up as far as I know. :frowning:

Note that you don't need to do the rclone mkdir - the rclone copy will do it for you.

I have a huge number of albums.
I know that the album is new, why look up the album? I just need to create a new one without checking if it already exists.

Thank you for the tip!

That explains the delay.

Rclone needs to see if the album exists already to avoid creating it again.

What you can do is use rclone rcd to set up an persistent rclone then use rclone rc sync/copy to upload the albums.

If you are doing a whole series of albums this will cache the album listings for 5 minutes.

Thank you for the tip, I did'n know the rcd mode.

Is it possible to change the expiration time to one hour? My workflow may require some days to finish.

Not currently, but this is a requested feature which will go into 1.55 I think

If it were possible it would be very useful for me to customise the cache expiration time.
Thank you!

The default is here if you want to patch the code

It really needs to be on a flag though!