Google Photos sync failed for multi-lined album names

Suspected Bug
1

What is the problem you are having with rclone?

Seems some album names are multi-lined and rclone is not able to sync these.

What is your rclone version (output from rclone version)

$ rclone --version
rclone v1.56.0
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.4.0-80-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.16.5
- go/linking: static
- go/tags: none
$

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Ubuntu 20.04 LTS 64-bit

Which cloud storage system are you using? (eg Google Drive)

Google Photos

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync --transfers=10 --max-duration=59m --retries 1 --timeout 1m --fast-list --gphotos-include-archived --gphotos-start-year=1970 --log-level INFO --log-file=photos_2021-07-26_16-02_shared-album.txt gphotos:shared-album /REDACTED_FULL_PATH/shared-album

Problem

I noticed this at the top of the log file:

2021/07/26 16:02:01 INFO  : Local file system at /REDACTED_FULL_PATH/shared-album: Transfer session deadline: 2021/07/26 17:01:01
2021/07/26 16:02:46 ERROR : Upahara Darshini (Barjari Oota) & UD Residency

Jayanagar Bangalore with REDACTED and REDACTED : error reading source directory: directory not found
2021/07/26 16:02:46 ERROR : California Citrus State Historic Park

 29 Dec 2019: error reading source directory: directory not found
2021/07/26 16:02:50 NOTICE: [TRUNCATED]

These two are shared albums shared with me by someone else so I don't control the album name. I looked them up in the Google Photos web UI and they seem to look normal. However it seems the album name is multi lined based on the log (some \n I think?). I confirmed this by running:

$ rclone lsd gphotos:shared-album
          -1 2021-07-26 20:13:26        14 10TB Seagate EXOS Purchase
[TRUNCATED]
          -1 2021-07-26 20:13:26        67 California Citrus State Historic Park

 29 Dec 2019
          -1 2021-07-26 20:13:26        30 Cambramatta for the Moon Festival
[TRUNCATED]
          -1 2021-07-26 20:13:26        97 UYN Community Masks Ordering - July 2020
          -1 2021-07-26 20:13:26        19 Upahara Darshini (Barjari Oota) & UD Residency

Jayanagar Bangalore with Ushaji Savithri and Sheela 
          -1 2021-07-26 20:13:26       299 Van Gogh and the Seasons - NGV Exhibition
[TRUNCATED]
 $

My attempt to create an album that has a multi-lined album name has failed (at least via the Google Photos web UI). These two albums are a few years old so it could be that a bug in the past allowed creating albums with multi-line names and that is no longer possible.

Next step

Seems rclone needs to handle the case where an album name is multi-lined?

2

This seems like a bug. I'd really need a way of creating an album with \n in if possible to fix.

Rclone can deal with names with \n in but I'm not sure the google photos API can...

3

Yeah I'm not sure how to create an album with \n either :confused:

Just to triple check what's going on, I'm going to run the command with -vv and --dump bodies similar to how you suggested to debug in https://forum.rclone.org/t/google-photos-downloading-into-wrong-by-month-folder/25641/6:
rclone lsd -vv --dump bodies gphotos:shared-album --log-file check-multi-lined-albun-name.log

4

This was useful in confirming that there are indeed \n in these two album names:

    {    
      "id": "_REDACTED_", 
      "title": "California Citrus State Historic Park\n\n 29 Dec 2019",                                
      "productUrl": "https://photos.google.com/lr/album/_REDACTED_",
      "mediaItemsCount": "67",
      "coverPhotoBaseUrl": "https://lh3.googleusercontent.com/lr/_REDACTED_",
      "coverPhotoMediaItemId": "_REDACTED_"
    }, 

    {
      "id": "_REDACTED_",
      "title": "Upahara Darshini (Barjari Oota) & UD Residency\n\nJayanagar Bangalore with name 1 and name2 ",                                                                                      
      "productUrl": "https://photos.google.com/lr/album/_REDACTED_",
      "mediaItemsCount": "19",
      "coverPhotoBaseUrl": "https://lh3.googleusercontent.com/lr/_REDACTED_",
      "coverPhotoMediaItemId": "_REDACTED_"
    },

@ncw it might be a simple matter of removing the \n when rclone sees them because I assume when rclone tries to create a folder with this name it fails the local file system will complain. I think the original error of error reading source directory: directory not found is because rclone sync is looking for a local directory to compare against and can't find it due to the \n?

5

I don't know if you fancy having a go at this, but the place to do this would be here

Something like

album.Title = strings.Replace(album.Title, "/n", " ", -1)

Though it might actually be better to use rclone's encoding framework here which will translate all control characters and other awkward characters into safe unicode equivalents...

6

Thanks for the reference - I'll have a go this weekend. This error is bugging me so I have an extra incentive to get this fixed :slight_smile:

7

@ncw I have just sent googlephotos: Use encoder for album names by timevortex · Pull Request #5535 · rclone/rclone · GitHub attempting to use rclone's encoding framework. Seems to work in my testing so far.

8

Closing this as the fix for this has been merged into the master branch!

1 Like
9

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.