Google Photos backend creating hundreds of single-photo albums

What is the problem you are having with rclone?

When syncing to Google Photos "album" location, it creates hundreds of albums containing one photo each, instead of an album with all the photos in for each directory as I would expect. Some albums appear to contain the right number of albums, but I now have hundreds of erroneous albums, with no bulk-delete function available. :flushed:

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.2
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.4.0-137-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.18.6
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Photos.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync ~/Backups/Exports photos:album --log-level DEBUG --log-file ~/rclone/photos.txt --tpslimit 1

My Exports directory is structured as follows:

tree | tail -n 100
# ...
β”œβ”€β”€ 2022-10-28 Blahblah
β”‚   β”œβ”€β”€ DSCF4644.jpeg
β”‚   β”œβ”€β”€ DSCF4646.jpeg
β”‚   β”œβ”€β”€ DSCF4650.jpeg
β”‚   β”œβ”€β”€ DSCF4695.jpeg
β”‚   β”œβ”€β”€ DSCF4697.jpeg
β”‚   β”œβ”€β”€ DSCF4727.jpeg
β”‚   β”œβ”€β”€ DSCF4731.jpeg
β”‚   └── DSCF4738.jpeg
β”œβ”€β”€ 2022-11-03 Blahblah
β”‚   β”œβ”€β”€ DSCF4782.jpeg
β”‚   β”œβ”€β”€ DSCF4797.jpeg
β”‚   β”œβ”€β”€ DSCF4829.jpeg
β”‚   └── DSCF4845.jpeg
β”œβ”€β”€ 2022-11-27 Blahblah
β”‚   β”œβ”€β”€ DSCF4882.jpeg
β”‚   β”œβ”€β”€ DSCF4891.jpeg
β”‚   β”œβ”€β”€ DSCF4893.jpeg
β”‚   β”œβ”€β”€ DSCF4899.jpeg
β”‚   β”œβ”€β”€ DSCF4900.jpeg
β”‚   β”œβ”€β”€ DSCF4903.jpeg
β”‚   β”œβ”€β”€ DSCF4906.jpeg
β”‚   β”œβ”€β”€ DSCF4918.jpeg
β”‚   └── DSCF4951.jpeg

So I would expect each single directory to create one single album containing all the photos within that directory.

The rclone config contents with secrets removed.

rclone config show photos
type = google photos
token = {"access_token":"xxx","refresh_token":"xxx","expiry":"2023-01-23T23:01:06.189769934Z"}

A log from the command with the -vv flag

:link: Logs.

I killed the rsync job here as it had already created 3 new albums (per watch 'rclone lsd photos:album | grep \{ | wc -l'), and there's no easy way to mass-delete albums on Google Photos.

That’s an older version.

Can you update to the latest and retest?

Hundreds? That is weird. You can try --transfers 1 to see if that helps.

Can you do a small test and show a -vv log of the problem?

Sure, I'll re-test on the latest version and post a new log below.

Here's it occurring on the latest version with --transfers 1. I'm up to 460 albums in Google Photos now, from a directory containing 60 photo folders :smile:

:link: Log from rclone v1.61.1

I've found the cause for this.

It is rclone's duplicate directory detection...

You'll see if you do an rclone lsf of the albums directory, you'll albums like this

test_album {ABdUxBg6PNE4kAXDxKVxIZM-_G_rs8xEtPi80_0lvXHQeDGAgTNaU9T6vnw-HIgQJ1nZQ1md2EdQ}/
test_album {ABdUxBgFDDy2wPdnlg51YF7j7OHe5VoX-qI9piCGPhkTEFQ8HK4Iase6F1JJenuoWunTidR_cfs-}/
test_album {ABdUxBhkUi1x9KYuSHya_ajTeTPpNw0UOXrfebtuyjgcFnki-L0SkkHlSoeBvrfuEo22MF5zEQ1B}/
test_album {ABdUxBipkIsxZAafvE-RnYdLLTHUTvZwMJWB2UPDxEReb3R_bDeesZ5BGjL2CUHNx5Ced__O_9Qu}/
test_album {ABdUxBjdNfxqO5kIFiAmkQecQzuTRB-mmWXX66Lgvzmnc43qY3fDniuA2sXVTwaiPWQRZcoEUhWC}/

The {XXX} are to disambiguate identical names.

However there is no test_album as rclone lsf will show you.

So if you upload to test_album rclone will create a new suffixed album.

So I think if you had no duplicate folders everything would work just fine.

I don't know how the first duplicate folder got in there though.

Rclone's handling of uploads to a potentially duplicated folder could be better! If you upload to test_album it could

  • give an error saying there are duplicate album names
  • pick one and upload there

Rather than what it does at the moment

  • create a new album!

Awesome, thank you @ncw - I can confirm that after deleting every album (yay macros!), and re-running rclone on an "empty" output, there are no duplicates.

As to where the duplicates came from, it looks like I already had some albums by the same names from long ago, so that might have confused rclone :thinking: Not sure.

Either way, if anyone has the same issue, deleting the albums with duplicate names does indeed resolve the issue.

Thanks all!

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.