Sync Google Photos Issues - Src Chnage & No Dest Deletes

Bones · December 26, 2020, 5:09am

What is the problem you are having with rclone?

I cant seem to control sync to Google Photos - two apparent issues

A Change to the Source File is not causing the file to be uploaded on next Sync (nothing to Transfer...)
When -I ( transfer all files) is used the result is Duplicates in GP (and Dedupe will not remove them as names are different...)

If i am missing some setup - or the way it should work i am sorry - happy to adjust/redo
Hope the info below gives enough details

In summary my objective is to be able to Sync a folder structure to Google Photos with the occational new image or update Exif data within existing images (ie small text change...)

Any help appreciated

What is your rclone version (output from `rclone version`)

rclone v1.53.3

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Synology DSM 6.2 64 Bit

Which cloud storage system are you using? (eg Google Drive)

Google Phots

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone sync "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum" RawPhotosG:album -P -v -I

The rclone config contents with secrets removed.

Tony.Watkins@NAS49:~$ rclone config show RawPhotosG
--------------------
[RawPhotosG]
type = google photos
token = {"access_token":"","token_type":"Bearer","refresh_token":"","expiry":"2020-12-26T12:28:33.388310213+08:00"}

A log from the command with the `-vv` flag

I have included the logs from the various commands that i think demonstrates the challenge. In summary

Delete all items from GP account
Sync one Image
Edit image
re-do Sync - observe no transfer
force Update - observe duplicates

Tony.Watkins@NAS49:~$
Tony.Watkins@NAS49:~$ Rem - Delete all items from GP (Images & Albums)
-sh: syntax error near unexpected token `('
Tony.Watkins@NAS49:~$ rclone sync "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum" RawPhotosG:album -P -vv
2020/12/26 12:40:22 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "sync" "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum" "RawPhotosG:album" "-P" "-vv"]
2020/12/26 12:40:22 DEBUG : Creating backend with remote "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum"
2020/12/26 12:40:22 DEBUG : Using config file from "/var/services/homes/Tony.Watkins/.config/rclone/rclone.conf"
2020/12/26 12:40:22 DEBUG : Creating backend with remote "RawPhotosG:album"
2020-12-26 12:40:22 DEBUG : Google Photos path "album": List: dir=""
2020-12-26 12:40:23 DEBUG : Google Photos path "album": >List: err=<nil>
2020-12-26 12:40:23 DEBUG : Google Photos path "album": Waiting for checks to finish
2020-12-26 12:40:23 DEBUG : Google Photos path "album": Waiting for transfers to finish
2020-12-26 12:40:23 DEBUG : Google Photos path "album": Put: src=Pics1/DSC10840.JPG
2020-12-26 12:40:23 DEBUG : Pics1/DSC10840.JPG: Update: src=Pics1/DSC10840.JPG
2020-12-26 12:40:32 DEBUG : Pics1/DSC10840.JPG: >Update: err=<nil>
2020-12-26 12:40:32 DEBUG : Google Photos path "album": >Put:
2020-12-26 12:40:32 DEBUG : Pics1/DSC10840.JPG: Size:
2020-12-26 12:40:32 DEBUG : Pics1/DSC10840.JPG: >Size:
2020-12-26 12:40:32 INFO  : Pics1/DSC10840.JPG: Copied (new)
2020-12-26 12:40:32 DEBUG : Waiting for deletions to finish
Transferred:        9.316M / 9.316 MBytes, 100%, 1003.482 kBytes/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:        10.3s
2020/12/26 12:40:32 INFO  :
Transferred:        9.316M / 9.316 MBytes, 100%, 1003.482 kBytes/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:        10.3s

2020/12/26 12:40:32 DEBUG : 3 go routines active
Tony.Watkins@NAS49:~$ Rem - result - image in Album - correct - then Edit EXIF Description (Change 12:41:30) confirm Date on file has changed (12:41:30 PM)
-sh: syntax error near unexpected token `('
Tony.Watkins@NAS49:~$ rclone sync "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum" RawPhotosG:album -P -vv
2020/12/26 12:45:28 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "sync" "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum" "RawPhotosG:album" "-P" "-vv"]
2020/12/26 12:45:28 DEBUG : Creating backend with remote "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum"
2020/12/26 12:45:28 DEBUG : Using config file from "/var/services/homes/Tony.Watkins/.config/rclone/rclone.conf"
2020/12/26 12:45:28 DEBUG : Creating backend with remote "RawPhotosG:album"
2020-12-26 12:45:28 DEBUG : Google Photos path "album": List: dir=""
2020-12-26 12:45:30 DEBUG : Google Photos path "album": >List: err=<nil>
2020-12-26 12:45:30 DEBUG : Google Photos path "album": List: dir="Pics1"
2020-12-26 12:45:31 DEBUG : Google Photos path "album": >List: err=<nil>
2020-12-26 12:45:31 DEBUG : Pics1/DSC10840.JPG: Size:
2020-12-26 12:45:31 DEBUG : Pics1/DSC10840.JPG: >Size:
2020-12-26 12:45:31 DEBUG : Pics1/DSC10840.JPG: Size:
2020-12-26 12:45:31 DEBUG : Pics1/DSC10840.JPG: >Size:
2020-12-26 12:45:31 DEBUG : Google Photos path "album": Waiting for checks to finish
2020-12-26 12:45:31 DEBUG : Pics1/DSC10840.JPG: Sizes identical
2020-12-26 12:45:31 DEBUG : Pics1/DSC10840.JPG: Unchanged skipping
2020-12-26 12:45:31 DEBUG : Google Photos path "album": Waiting for transfers to finish
2020-12-26 12:45:31 DEBUG : Waiting for deletions to finish
2020-12-26 12:45:31 INFO  : There was nothing to transfer
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks:                 1 / 1, 100%
Elapsed time:         2.8s
2020/12/26 12:45:31 INFO  :
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks:                 1 / 1, 100%
Elapsed time:         2.8s

2020/12/26 12:45:31 DEBUG : 5 go routines active
Tony.Watkins@NAS49:~$ Rem - conlusion is EXIF change is not effecting size on Disk - so need to force Update with -I ()
-sh: syntax error near unexpected token `('
Tony.Watkins@NAS49:~$ Rem - result - image in Album - correct - then Edit EXIF Description (Change 12:46:45) confirm Date on file has changed (12:46:45 PM) diskspace static at 9.31 MB (9,768,960 bytes)
-sh: syntax error near unexpected token `('
Tony.Watkins@NAS49:~$ rclone sync "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum" RawPhotosG:album -P -vv -I
2020/12/26 12:50:36 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "sync" "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum" "RawPhotosG:album" "-P" "-vv" "-I"]
2020/12/26 12:50:36 DEBUG : Creating backend with remote "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum"
2020/12/26 12:50:36 DEBUG : Using config file from "/var/services/homes/Tony.Watkins/.config/rclone/rclone.conf"
2020/12/26 12:50:36 DEBUG : Creating backend with remote "RawPhotosG:album"
2020-12-26 12:50:36 DEBUG : Google Photos path "album": List: dir=""
2020-12-26 12:50:38 DEBUG : Google Photos path "album": >List: err=<nil>
2020-12-26 12:50:38 DEBUG : Google Photos path "album": List: dir="Pics1"
2020-12-26 12:50:40 DEBUG : Google Photos path "album": >List: err=<nil>
2020-12-26 12:50:40 DEBUG : Pics1/DSC10840.JPG: Size:
2020-12-26 12:50:40 DEBUG : Pics1/DSC10840.JPG: >Size:
2020-12-26 12:50:40 DEBUG : Pics1/DSC10840.JPG: Transferring unconditionally as --ignore-times is in use
2020-12-26 12:50:40 DEBUG : Pics1/DSC10840.JPG: Update: src=Pics1/DSC10840.JPG
2020-12-26 12:50:40 DEBUG : Google Photos path "album": Waiting for checks to finish
2020-12-26 12:50:40 DEBUG : Google Photos path "album": Waiting for transfers to finish
2020-12-26 12:50:47 DEBUG : Pics1/DSC10840.JPG: >Update: err=<nil>
2020-12-26 12:50:47 DEBUG : Pics1/DSC10840.JPG: Size:
2020-12-26 12:50:47 DEBUG : Pics1/DSC10840.JPG: >Size:
2020-12-26 12:50:47 INFO  : Pics1/DSC10840.JPG: Copied (replaced existing)
2020-12-26 12:50:47 DEBUG : Waiting for deletions to finish
Transferred:        9.316M / 9.316 MBytes, 100%, 1.293 MBytes/s, ETA 0s
Checks:                 1 / 1, 100%
Transferred:            1 / 1, 100%
Elapsed time:        11.1s
2020/12/26 12:50:47 INFO  :
Transferred:        9.316M / 9.316 MBytes, 100%, 1.293 MBytes/s, ETA 0s
Checks:                 1 / 1, 100%
Transferred:            1 / 1, 100%
Elapsed time:        11.1s

2020/12/26 12:50:47 DEBUG : 3 go routines active
Tony.Watkins@NAS49:~$ rclone lsl RawPhotosG:album
       -1 2016-05-17 09:20:36.000000000 Pics1/DSC10840 {AP0gtnfTQNSpaSW84v54meMLxhoGTZDQ-KEvgxYvr0tqD8WOCQExdneSy9CoQuBdea0sw1TKLjnDW-35_n2Eenc2cdiHhO_s2g}.JPG
       -1 2016-05-17 09:20:36.000000000 Pics1/DSC10840 {AP0gtnfdbivquguzHzQmovSr4NbkkaXMs3jBTFPtisEefq37Z3GUKt4g-dhi33t87wj8AKtb2jIFm4l9jxuwZoxXmYWOaEmodA}.JPG
Tony.Watkins@NAS49:~$ Rem - now have duplicates in GP ()
-sh: syntax error near unexpected token `('
Tony.Watkins@NAS49:~$ rclone sync "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum" RawPhotosG:album -P -vv -I
2020/12/26 12:52:55 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "sync" "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum" "RawPhotosG:album" "-P" "-vv" "-I"]
2020/12/26 12:52:55 DEBUG : Creating backend with remote "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum"
2020/12/26 12:52:55 DEBUG : Using config file from "/var/services/homes/Tony.Watkins/.config/rclone/rclone.conf"
2020/12/26 12:52:55 DEBUG : Creating backend with remote "RawPhotosG:album"
2020-12-26 12:52:55 DEBUG : Google Photos path "album": List: dir=""
2020-12-26 12:52:56 DEBUG : Google Photos path "album": >List: err=<nil>
2020-12-26 12:52:56 DEBUG : Google Photos path "album": List: dir="Pics1"
2020-12-26 12:52:57 DEBUG : Google Photos path "album": >List: err=<nil>
2020-12-26 12:52:57 DEBUG : Pics1/DSC10840 {AP0gtnfdbivquguzHzQmovSr4NbkkaXMs3jBTFPtisEefq37Z3GUKt4g-dhi33t87wj8AKtb2jIFm4l9jxuwZoxXmYWOaEmodA}.JPG: Size:
2020-12-26 12:52:57 DEBUG : Pics1/DSC10840 {AP0gtnfdbivquguzHzQmovSr4NbkkaXMs3jBTFPtisEefq37Z3GUKt4g-dhi33t87wj8AKtb2jIFm4l9jxuwZoxXmYWOaEmodA}.JPG: >Size:
2020-12-26 12:52:57 DEBUG : Pics1/DSC10840 {AP0gtnfTQNSpaSW84v54meMLxhoGTZDQ-KEvgxYvr0tqD8WOCQExdneSy9CoQuBdea0sw1TKLjnDW-35_n2Eenc2cdiHhO_s2g}.JPG: Size:
2020-12-26 12:52:57 DEBUG : Pics1/DSC10840 {AP0gtnfTQNSpaSW84v54meMLxhoGTZDQ-KEvgxYvr0tqD8WOCQExdneSy9CoQuBdea0sw1TKLjnDW-35_n2Eenc2cdiHhO_s2g}.JPG: >Size:
2020-12-26 12:52:57 DEBUG : Google Photos path "album": Put: src=Pics1/DSC10840.JPG
2020-12-26 12:52:57 DEBUG : Pics1/DSC10840.JPG: Update: src=Pics1/DSC10840.JPG
2020-12-26 12:52:57 DEBUG : Google Photos path "album": Waiting for checks to finish
2020-12-26 12:52:57 DEBUG : Google Photos path "album": Waiting for transfers to finish
2020-12-26 12:53:05 DEBUG : Pics1/DSC10840.JPG: >Update: err=<nil>
2020-12-26 12:53:05 DEBUG : Google Photos path "album": >Put:
2020-12-26 12:53:05 DEBUG : Pics1/DSC10840.JPG: Size:
2020-12-26 12:53:05 DEBUG : Pics1/DSC10840.JPG: >Size:
2020-12-26 12:53:05 INFO  : Pics1/DSC10840.JPG: Copied (new)
2020-12-26 12:53:05 DEBUG : Waiting for deletions to finish
2020-12-26 12:53:05 DEBUG : Pics1/DSC10840 {AP0gtnfTQNSpaSW84v54meMLxhoGTZDQ-KEvgxYvr0tqD8WOCQExdneSy9CoQuBdea0sw1TKLjnDW-35_n2Eenc2cdiHhO_s2g}.JPG: Size:
2020-12-26 12:53:05 DEBUG : Pics1/DSC10840 {AP0gtnfTQNSpaSW84v54meMLxhoGTZDQ-KEvgxYvr0tqD8WOCQExdneSy9CoQuBdea0sw1TKLjnDW-35_n2Eenc2cdiHhO_s2g}.JPG: >Size:
2020-12-26 12:53:05 DEBUG : Pics1/DSC10840 {AP0gtnfdbivquguzHzQmovSr4NbkkaXMs3jBTFPtisEefq37Z3GUKt4g-dhi33t87wj8AKtb2jIFm4l9jxuwZoxXmYWOaEmodA}.JPG: Size:
2020-12-26 12:53:05 DEBUG : Pics1/DSC10840 {AP0gtnfdbivquguzHzQmovSr4NbkkaXMs3jBTFPtisEefq37Z3GUKt4g-dhi33t87wj8AKtb2jIFm4l9jxuwZoxXmYWOaEmodA}.JPG: >Size:
2020-12-26 12:53:07 INFO  : Pics1/DSC10840 {AP0gtnfdbivquguzHzQmovSr4NbkkaXMs3jBTFPtisEefq37Z3GUKt4g-dhi33t87wj8AKtb2jIFm4l9jxuwZoxXmYWOaEmodA}.JPG: Deleted
2020-12-26 12:53:08 INFO  : Pics1/DSC10840 {AP0gtnfTQNSpaSW84v54meMLxhoGTZDQ-KEvgxYvr0tqD8WOCQExdneSy9CoQuBdea0sw1TKLjnDW-35_n2Eenc2cdiHhO_s2g}.JPG: Deleted
Transferred:        9.316M / 9.316 MBytes, 100%, 838.973 kBytes/s, ETA 0s
Checks:                 2 / 2, 100%
Deleted:                2
Transferred:            1 / 1, 100%
Elapsed time:        13.0s
2020/12/26 12:53:08 INFO  :
Transferred:        9.316M / 9.316 MBytes, 100%, 838.973 kBytes/s, ETA 0s
Checks:                 2 / 2, 100%
Deleted:                2
Transferred:            1 / 1, 100%
Elapsed time:        13.0s

2020/12/26 12:53:08 DEBUG : 4 go routines active
Tony.Watkins@NAS49:~$ Rem - still have duplicate in GP (ie via web page...)
-sh: syntax error near unexpected token `('
Tony.Watkins@NAS49:~$ rclone lsl RawPhotosG:album
Tony.Watkins@NAS49:~$ rclone lsl RawPhotosG:album
Tony.Watkins@NAS49:~$ rclone lsl RawPhotosG:
2020/12/26 12:55:19 ERROR : upload: error listing: directory not found
       -1 2016-05-17 09:20:36.000000000 media/all/DSC10840 {AP0gtnfTQNSpaSW84v54meMLxhoGTZDQ-KEvgxYvr0tqD8WOCQExdneSy9CoQuBdea0sw1TKLjnDW-35_n2Eenc2cdiHhO_s2g}.JPG
       -1 2016-05-17 09:20:36.000000000 media/all/DSC10840 {AP0gtnfdbivquguzHzQmovSr4NbkkaXMs3jBTFPtisEefq37Z3GUKt4g-dhi33t87wj8AKtb2jIFm4l9jxuwZoxXmYWOaEmodA}.JPG
       -1 2016-05-17 09:20:36.000000000 media/by-year/2016/DSC10840 {AP0gtnfTQNSpaSW84v54meMLxhoGTZDQ-KEvgxYvr0tqD8WOCQExdneSy9CoQuBdea0sw1TKLjnDW-35_n2Eenc2cdiHhO_s2g}.JPG
       -1 2016-05-17 09:20:36.000000000 media/by-year/2016/DSC10840 {AP0gtnfdbivquguzHzQmovSr4NbkkaXMs3jBTFPtisEefq37Z3GUKt4g-dhi33t87wj8AKtb2jIFm4l9jxuwZoxXmYWOaEmodA}.JPG
^C
Tony.Watkins@NAS49:~$ Rem - rclone seemed confused - lsl not returning and error on 3rd atempt - had to ctrlC () - but have no images in :album/Pics1 - but duplicates in "Photos"... all strange
-sh: syntax error near unexpected token `('
Tony.Watkins@NAS49:~$ rclone sync "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum" RawPhotosG:album -P -vv -I
2020/12/26 12:58:40 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "sync" "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum" "RawPhotosG:album" "-P" "-vv" "-I"]
2020/12/26 12:58:40 DEBUG : Creating backend with remote "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum"
2020/12/26 12:58:40 DEBUG : Using config file from "/var/services/homes/Tony.Watkins/.config/rclone/rclone.conf"
2020/12/26 12:58:40 DEBUG : Creating backend with remote "RawPhotosG:album"
2020-12-26 12:58:40 DEBUG : Google Photos path "album": List: dir=""
2020-12-26 12:58:43 DEBUG : Google Photos path "album": Error reading media count: strconv.ParseInt: parsing "": invalid syntax
2020-12-26 12:58:43 DEBUG : Google Photos path "album": >List: err=<nil>
2020-12-26 12:58:43 DEBUG : Google Photos path "album": List: dir="Pics1"
2020-12-26 12:58:44 DEBUG : Google Photos path "album": >List: err=<nil>
2020-12-26 12:58:44 DEBUG : Google Photos path "album": Waiting for checks to finish
2020-12-26 12:58:44 DEBUG : Google Photos path "album": Waiting for transfers to finish
2020-12-26 12:58:44 DEBUG : Google Photos path "album": Put: src=Pics1/DSC10840.JPG
2020-12-26 12:58:44 DEBUG : Pics1/DSC10840.JPG: Update: src=Pics1/DSC10840.JPG
2020-12-26 12:58:54 DEBUG : Pics1/DSC10840.JPG: >Update: err=<nil>
2020-12-26 12:58:54 DEBUG : Google Photos path "album": >Put:
2020-12-26 12:58:54 DEBUG : Pics1/DSC10840.JPG: Size:
2020-12-26 12:58:54 DEBUG : Pics1/DSC10840.JPG: >Size:
2020-12-26 12:58:54 INFO  : Pics1/DSC10840.JPG: Copied (new)
2020-12-26 12:58:54 DEBUG : Waiting for deletions to finish
Transferred:        9.316M / 9.316 MBytes, 100%, 1018.152 kBytes/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:        13.2s
2020/12/26 12:58:54 INFO  :
Transferred:        9.316M / 9.316 MBytes, 100%, 1018.152 kBytes/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:        13.2s

2020/12/26 12:58:54 DEBUG : 4 go routines active
Tony.Watkins@NAS49:~$ Rem - on Web two pics PHotos and one in album:Pics1
-sh: Rem: command not found
Tony.Watkins@NAS49:~$ rclone lsl RawPhotosG:album
       -1 2016-05-17 09:20:36.000000000 Pics1/DSC10840.JPG
Tony.Watkins@NAS49:~$ rclone lsl RawPhotosG:
2020/12/26 13:00:31 ERROR : upload: error listing: directory not found
       -1 2016-05-17 09:20:36.000000000 media/by-year/2016/DSC10840 {AP0gtnfTQNSpaSW84v54meMLxhoGTZDQ-KEvgxYvr0tqD8WOCQExdneSy9CoQuBdea0sw1TKLjnDW-35_n2Eenc2cdiHhO_s2g}.JPG
       -1 2016-05-17 09:20:36.000000000 media/by-year/2016/DSC10840 {AP0gtnfdbivquguzHzQmovSr4NbkkaXMs3jBTFPtisEefq37Z3GUKt4g-dhi33t87wj8AKtb2jIFm4l9jxuwZoxXmYWOaEmodA}.JPG
       -1 2016-05-17 09:20:36.000000000 album/Pics1/DSC10840.JPG
       -1 2016-05-17 09:20:36.000000000 media/all/DSC10840 {AP0gtnfTQNSpaSW84v54meMLxhoGTZDQ-KEvgxYvr0tqD8WOCQExdneSy9CoQuBdea0sw1TKLjnDW-35_n2Eenc2cdiHhO_s2g}.JPG
       -1 2016-05-17 09:20:36.000000000 media/all/DSC10840 {AP0gtnfdbivquguzHzQmovSr4NbkkaXMs3jBTFPtisEefq37Z3GUKt4g-dhi33t87wj8AKtb2jIFm4l9jxuwZoxXmYWOaEmodA}.JPG
^C
Tony.Watkins@NAS49:~$

ncw · December 26, 2020, 2:27pm

Google photos doesn't support file lengths, dates or hashes so this is expected I think.

You can work around this with the update flag

 -u, --update                    Skip files that are newer on the destination.

I think --update is what you want here.

Bones · December 26, 2020, 4:37pm

Thanks - mmm that makes it tough - saw the hashes was out - missed the dates & Lenghts

I changed to
...rclone sync "/volume2/Files/AllOurMedia/GP_Checks/GPAlbum" RawPhotosG:album -P -vv --update...

Not getting Duplicates in Album - but am getting duplicate in "media/all" still

Am not understanding the --update operation as i thought i wanted to transfer all New ones in the source - but i dont need to understand when it works
If this work around does not do the job - can i set it to copy only files after a date in the source dir?

Thanks for you help

ncw · December 26, 2020, 4:44pm

The google photos can read the upload date (which is the exif date of the photo if available) it just can't set it, which makes it useless for syncing. --update is the work-around for this.

That is good

You can only sync to the album path, so not sure what you mean here?

You can use --min-age and --max-age which can take a date. Use rclone copy in this case not rclone sync

Bones · December 26, 2020, 5:01pm

GP has Albums and the general Media folders/all,/by-year, by-month..... - all these folders reported duplicates and the main "Photos" location on the Web page shows duplicates
... 2020/12/27 00:25:01 ERROR : upload: error listing: directory not found
-1 2016-05-17 09:20:36.000000000 album/Pics1/DSC10840.JPG
-1 2016-05-17 09:18:27.000000000 album/Pics1/DSC20924 - +12.JPG
-1 2016-05-17 09:20:36.000000000 media/all/DSC10840 {AP0gtnctMCGPnCcwk5nYOn_mi7q3Indpxha4F8Jn7efqijEXsK7o-FDGcESIZyHKRAAEgji6SsycCyuk7_n_xp0EOL_MM-T3-g}.JPG
-1 2016-05-17 09:20:36.000000000 media/all/DSC10840 {AP0gtneFwKRld6ShM61NTRxQxb-caqPYQVPO-Alfq6UfUDIOqhX3jB3n8EBxeUeOdxsTs_TFVjR_TOstpNLAFywpREuAewrWuA}.JPG
-1 2016-05-17 09:20:36.000000000 media/all/DSC10840 {AP0gtnetTqn_a7rdMsIDgJaGHcKRUgLz4yyaEOqHqcwmAIIXqF6KLWMph6iHDjoVS9untk6AwKFiUHS0USOoMlCtqdg4c-ECcQ}.JPG
-1 2016-05-17 09:18:27.000000000 media/all/DSC20924 - +12.JPG
-1 2016-05-17 09:20:36.000000000 media/by-year/2016/DSC10840 {AP0gtnctMCGPnCcwk5nYOn_mi7q3Indpxha4F8Jn7efqijEXsK7o-FDGcESIZyHKRAAEgji6SsycCyuk7_n_xp0EOL_MM-T3-g}.JPG
-1 2016-05-17 09:20:36.000000000 media/by-year/2016/DSC10840 {AP0gtneFwKRld6ShM61NTRxQxb-caqPYQVPO-Alfq6UfUDIOqhX3jB3n8EBxeUeOdxsTs_TFVjR_TOstpNLAFywpREuAewrWuA}.JPG
-1 2016-05-17 09:20:36.000000000 media/by-year/2016/DSC10840 {AP0gtnetTqn_a7rdMsIDgJaGHcKRUgLz4yyaEOqHqcwmAIIXqF6KLWMph6iHDjoVS9untk6AwKFiUHS0USOoMlCtqdg4c-ECcQ}.JPG
-1 2016-05-17 09:18:27.000000000 media/by-year/2016/DSC20924 - +12.JPG
-1 2016-05-17 09:20:36.000000000 media/by-month/2016/2016-05/DSC10840 {AP0gtnctMCGPnCcwk5nYOn_mi7q3Indpxha4F8Jn7efqijEXsK7o-FDGcESIZyHKRAAEgji6SsycCyuk7_n_xp0EOL_MM-T3-g}.JPG...

Saw the comment --min/max age - thanks

ncw · December 26, 2020, 8:05pm

Hmm...

Did the duplicates get uploaded without --update?

Bones · December 27, 2020, 1:22am

I have 2 files in the source - 1 i update, 1 i do not
I deleted the duplicates (leaving 1 instance of each image), updated the source file and
ran rclone sync source RawPhotosG:album -P -vv - result was "There was nothing to transfer" as expected

Updated Source File and ran with -P -vv -I

this gave me a duplicate of update image in the album and in media/all...

Updated Source File and ran with -P -vv --update

this removed the dup from the album - but left the dup in media/all.

running rclone --max-age 30m delete RawPhotosG:media -P -vv
seemed to exclude all files in media/all from deletion regardless of the max age value

2020-12-27 09:19:17 DEBUG : DSC10840 {AP0gtnctMCGPnCcwk5nYOn_mi7q3Indpxha4F8Jn7efqijEXsK7o-FDGcESIZyHKRAAEgji6SsycCyuk7_n_xp0EOL_MM-T3-g}.JPG: ModTime:
2020-12-27 09:19:17 DEBUG : DSC10840 {AP0gtnctMCGPnCcwk5nYOn_mi7q3Indpxha4F8Jn7efqijEXsK7o-FDGcESIZyHKRAAEgji6SsycCyuk7_n_xp0EOL_MM-T3-g}.JPG: >ModTime:
2020-12-27 09:19:17 DEBUG : DSC10840 {AP0gtnctMCGPnCcwk5nYOn_mi7q3Indpxha4F8Jn7efqijEXsK7o-FDGcESIZyHKRAAEgji6SsycCyuk7_n_xp0EOL_MM-T3-g}.JPG: Size:
2020-12-27 09:19:17 DEBUG : DSC10840 {AP0gtnctMCGPnCcwk5nYOn_mi7q3Indpxha4F8Jn7efqijEXsK7o-FDGcESIZyHKRAAEgji6SsycCyuk7_n_xp0EOL_MM-T3-g}.JPG: >Size:
2020-12-27 09:19:17 DEBUG : DSC10840 {AP0gtnctMCGPnCcwk5nYOn_mi7q3Indpxha4F8Jn7efqijEXsK7o-FDGcESIZyHKRAAEgji6SsycCyuk7_n_xp0EOL_MM-T3-g}.JPG: Excluded
2020-12-27 09:19:17 DEBUG : DSC10840 {AP0gtneHN9H0Ge6IA5Miac_QOZ6EX6bWBfEdlVdMd1raLOiItqQ8FgT7KIuMvXkMKKY23KKWySBQ75ipbYwd7kLRiNWa9VK1cg}.JPG: ModTime:
2020-12-27 09:19:17 DEBUG : DSC10840 {AP0gtneHN9H0Ge6IA5Miac_QOZ6EX6bWBfEdlVdMd1raLOiItqQ8FgT7KIuMvXkMKKY23KKWySBQ75ipbYwd7kLRiNWa9VK1cg}.JPG: >ModTime:
2020-12-27 09:19:17 DEBUG : DSC10840 {AP0gtneHN9H0Ge6IA5Miac_QOZ6EX6bWBfEdlVdMd1raLOiItqQ8FgT7KIuMvXkMKKY23KKWySBQ75ipbYwd7kLRiNWa9VK1cg}.JPG: Size:
2020-12-27 09:19:17 DEBUG : DSC10840 {AP0gtneHN9H0Ge6IA5Miac_QOZ6EX6bWBfEdlVdMd1raLOiItqQ8FgT7KIuMvXkMKKY23KKWySBQ75ipbYwd7kLRiNWa9VK1cg}.JPG: >Size:
2020-12-27 09:19:17 DEBUG : DSC10840 {AP0gtneHN9H0Ge6IA5Miac_QOZ6EX6bWBfEdlVdMd1raLOiItqQ8FgT7KIuMvXkMKKY23KKWySBQ75ipbYwd7kLRiNWa9VK1cg}.JPG: Excluded
2020-12-27 09:19:17 DEBUG : DSC10840 {AP0gtnfRv_hgT8oZVPbFTn8aIL7kuvE10dwDIKDS3NY6h7-T5VaFIqaU6u8TNc2mnFg-u2820Kcw0ePKxbC3zmeGxodlGfhMsQ}.JPG: ModTime:
2020-12-27 09:19:17 DEBUG : DSC10840 {AP0gtnfRv_hgT8oZVPbFTn8aIL7kuvE10dwDIKDS3NY6h7-T5VaFIqaU6u8TNc2mnFg-u2820Kcw0ePKxbC3zmeGxodlGfhMsQ}.JPG: >ModTime:
2020-12-27 09:19:17 DEBUG : DSC10840 {AP0gtnfRv_hgT8oZVPbFTn8aIL7kuvE10dwDIKDS3NY6h7-T5VaFIqaU6u8TNc2mnFg-u2820Kcw0ePKxbC3zmeGxodlGfhMsQ}.JPG: Size:
2020-12-27 09:19:17 DEBUG : DSC10840 {AP0gtnfRv_hgT8oZVPbFTn8aIL7kuvE10dwDIKDS3NY6h7-T5VaFIqaU6u8TNc2mnFg-u2820Kcw0ePKxbC3zmeGxodlGfhMsQ}.JPG: >Size:
2020-12-27 09:19:17 DEBUG : DSC10840 {AP0gtnfRv_hgT8oZVPbFTn8aIL7kuvE10dwDIKDS3NY6h7-T5VaFIqaU6u8TNc2mnFg-u2820Kcw0ePKxbC3zmeGxodlGfhMsQ}.JPG: Excluded
2020-12-27 09:19:17 DEBUG : DSC20924 - +12.JPG: ModTime:
2020-12-27 09:19:17 DEBUG : DSC20924 - +12.JPG: >ModTime:
2020-12-27 09:19:17 DEBUG : DSC20924 - +12.JPG: Size:
2020-12-27 09:19:17 DEBUG : DSC20924 - +12.JPG: >Size:
2020-12-27 09:19:17 DEBUG : DSC20924 - +12.JPG: Excluded

ncw · December 28, 2020, 10:25am

I think this is working as expected now. You'll get duplicates unless you use --update. Rclone can't permanently delete files on gphotos so the duplicate is left in the medial/all - it is no longer in the album.

Rclone can only remove photos from albums, it can't delete photos. This is a limitation of the google photos API.

Bones · December 28, 2020, 4:15pm

Thanks ncw for the confirmations

Its about where i got to the API with its current features/bugs leaves us snookered.

is no practical way to "sync" to Google Photos as you end-up with a duplicate in media/all for every source file that gets a modification - shame

All potential work arounds i can scheme up depend on deletes from media/all - and that is an API bug marked as infeasible...

dead end.

btw - rclone is fantastic - thanks

ncw · December 29, 2020, 12:52pm

I think you are right.

If you never modify your images then you will be fine - that is how I treat my photo archive.

The google photos API is very annoying... I've seen a project which hooks into the chrome developer tools to drive the website directly for doing a sync from google photos: https://github.com/perkeep/gphotos-cdp

Rewriting the google photos backend to use a tool like that would fix all the problems.

system · February 28, 2021, 8:52am

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.