Corrupted on transfer size differ on Google Photos

What is the problem you are having with rclone?

Hey! I've been syncing my local photos with Google Photos using the following command:

/usr/local/bin/rclone --gphotos-read-size --low-level-retries 5 --exclude '.*{/**,}' sync '/Volumes/Photos/Gallery/2009/_' Photos:album/2009

I have the following error for ANY photo rclone tries to sync:
2022/10/31 11:30:39 ERROR : DSC01283.JPG: corrupted on transfer: sizes differ 1449027 vs 1437043

Photos are stored on an USB drive formatted with APFS

Can you please help?

BTW, great work, guys. This tool saved my life many times.

R.

Run the command 'rclone version' and share the full output of the command.

rclone v1.61.0-beta.6512.65987f597

  • os/version: darwin 11.7 (64 bit)
  • os/kernel: 20.6.0 (x86_64)
  • os/type: darwin
  • os/arch: amd64
  • go/version: go1.19.2
  • go/linking: dynamic
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Google Photos

The command you were trying to run (eg rclone copy /tmp remote:tmp)

/usr/local/bin/rclone --gphotos-read-size --low-level-retries 5 --exclude '.*{/**,}' sync '/Volumes/Photos/Gallery/2009/_' Photos:album/2009

The rclone config contents with secrets removed.

[Photos]
type = google photos
client_id = ####
client_secret = ####
token = {"access_token":"####","token_type":"Bearer","refresh_token":"####","expiry":"2022-10-31T12:01:13.51067+01:00"}

A log from the command with the -vv flag

2022/10/31 11:44:01 DEBUG : rclone: Version "v1.61.0-beta.6512.65987f597" starting with parameters ["/usr/local/bin/rclone" "-vv" "--gphotos-read-size" "--low-level-retries" "5" "--exclude" ".*{/**,}" "sync" "/Volumes/Photos/Gallery/2009/_" "Photos:album/2009"]
2022/10/31 11:44:01 DEBUG : Creating backend with remote "/Volumes/Photos/Gallery/2009/_"
2022/10/31 11:44:01 DEBUG : Using config file from "/Users/riccardo/.config/rclone/rclone.conf"
2022/10/31 11:44:01 DEBUG : Creating backend with remote "Photos:album/2009"
2022/10/31 11:44:01 DEBUG : Photos: detected overridden config - adding "{kLu1M}" suffix to name
2022/10/31 11:44:01 DEBUG : fs cache: renaming cache item "Photos:album/2009" to be canonical "Photos{kLu1M}:album/2009"
2022/10/31 11:44:01 DEBUG : Google Photos path "album/2009": List: dir=""
2022/10/31 11:44:05 DEBUG : Google Photos path "album/2009": >List: err=directory not found
2022/10/31 11:44:05 DEBUG : Google Photos path "album/2009": Waiting for checks to finish
2022/10/31 11:44:05 DEBUG : Google Photos path "album/2009": Waiting for transfers to finish
2022/10/31 11:44:05 DEBUG : Google Photos path "album/2009": Put: src=DSC01285.JPG
2022/10/31 11:44:05 DEBUG : Google Photos path "album/2009": Put: src=DSC01281.JPG
2022/10/31 11:44:05 DEBUG : DSC01281.JPG: Update: src=DSC01281.JPG
2022/10/31 11:44:05 DEBUG : Google Photos path "album/2009": Put: src=DSC01282.JPG
2022/10/31 11:44:05 DEBUG : DSC01282.JPG: Update: src=DSC01282.JPG
2022/10/31 11:44:05 DEBUG : DSC01285.JPG: Update: src=DSC01285.JPG
2022/10/31 11:44:05 DEBUG : Google Photos path "album/2009": Put: src=DSC01283.JPG
2022/10/31 11:44:05 DEBUG : DSC01283.JPG: Update: src=DSC01283.JPG
2022/10/31 11:44:10 DEBUG : DSC01282.JPG: >Update: err=<nil>
2022/10/31 11:44:10 DEBUG : Google Photos path "album/2009": >Put:
2022/10/31 11:44:10 DEBUG : DSC01282.JPG: Size:
2022/10/31 11:44:10 DEBUG : Google Photos path "album/2009": List: dir=""
2022/10/31 11:44:11 DEBUG : Google Photos path "album/2009": >List: err=<nil>
2022/10/31 11:44:11 DEBUG : DSC01282.JPG: >Size:
2022/10/31 11:44:11 DEBUG : DSC01282.JPG: Size:
2022/10/31 11:44:11 DEBUG : DSC01282.JPG: >Size:
2022/10/31 11:44:11 DEBUG : DSC01282.JPG: Size:
2022/10/31 11:44:11 DEBUG : DSC01282.JPG: >Size:
2022/10/31 11:44:11 ERROR : DSC01282.JPG: corrupted on transfer: sizes differ 1478959 vs 1466975
2022/10/31 11:44:11 INFO  : DSC01282.JPG: Removing failed copy
2022/10/31 11:44:11 DEBUG : DSC01285.JPG: >Update: err=<nil>

I'm afraid reading sizes from google photos is a bit random. If you remove the above flag then the sync should work.

Yes, but how can it be a real sync if only the file name is checked and nothing else? For example size, checksum, etc.

Google photos API doesn't give you those things. In fact if you read the size of a image, it might read as a different size next time you read it.

In my opinion the Google photos API is very poor for syncing. Read the limitations section. The API is fine for uploading pics to google photos, but if you want them back in their original form then you'll need to use Google takeout.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.