Dates Incorrect on Videos Uploaded to Google Photos

mnivlac · February 10, 2021, 10:53pm

What is the problem you are having with rclone?

When uploading videos to Google Photos using rclone, the date on the video is the upload date instead of the date the video was taken. I understand from the documentation that "The date shown of media in Google Photos is the creation date as determined by the EXIF information, or the upload date if that is not known." When I upload the same video files to Google Photos using the Google-created "Backup and Sync" app on MacOS, the date correctly shows as the creation date of the video and not the upload date. The incorrect date behavior only occurs when using rclone. This leads me to believe that it is a problem with either the API or the rclone implementation, rather than the video files themselves.

What is your rclone version (output from `rclone version`)

rclone v1.53.3

os/arch: darwin/amd64
go version: go1.15.5

Which OS you are using and how many bits (eg Windows 7, 64 bit)

MacOS 11.2.1, 64-bit.

Which cloud storage system are you using? (eg Google Drive)

Google Photos

rclone -vv --log-file="/Users/m/code/projects/mediasort/VIDEOS/gphotos_log.txt" copy /Users/m/code/projects/mediasort/VIDEOS/TESTING/ gphotos:upload

The rclone config contents with secrets removed.

[gphotos]
type = google photos
read_only = false
token = {"access_token":"xxx","token_type":"Bearer","refresh_token":"xxxx","expiry":"2021-02-10T13:16:26.457739-07:00"}

A log from the command with the `-vv` flag

2021/02/10 12:16:26 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "-vv" "--log-file=/Users/m/code/projects/mediasort/VIDEOS/gphotos_log.txt" "copy" "/Users/m/code/projects/mediasort/VIDEOS/TESTING/" "gphotos:album"]
2021/02/10 12:16:26 DEBUG : Creating backend with remote "/Users/m/code/projects/mediasort/VIDEOS/TESTING/"
2021/02/10 12:16:26 DEBUG : Using config file from "/Users/m/.config/rclone/rclone.conf"
2021/02/10 12:16:26 DEBUG : Creating backend with remote "gphotos:album"
2021/02/10 12:16:26 DEBUG : Google Photos path "album": List: dir=""
2021/02/10 12:16:26 DEBUG : gphotos: Loaded invalid token from config file - ignoring
2021/02/10 12:16:27 DEBUG : gphotos: Saved new token in config file
2021/02/10 12:16:30 DEBUG : Google Photos path "album": >List: err=<nil>
2021/02/10 12:16:30 DEBUG : Google Photos path "album": Waiting for checks to finish
2021/02/10 12:16:30 DEBUG : Google Photos path "album": Waiting for transfers to finish
2021/02/10 12:16:30 DEBUG : Google Photos path "album": Put: src=DJI_0708_20201128_003902.MP4
2021/02/10 12:16:30 DEBUG : DJI_0708_20201128_003902.MP4: Update: src=DJI_0708_20201128_003902.MP4
2021/02/10 12:16:30 DEBUG : Google Photos path "album": Put: src=DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:16:30 DEBUG : DJI_0708_20201128_003902.MP4: >Update: err=can't upload files here
2021/02/10 12:16:30 DEBUG : DJI_0708_20201128_003902_copy.MP4: Update: src=DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:16:30 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:16:30 DEBUG : DJI_0708_20201128_003902_copy.MP4: >Update: err=can't upload files here
2021/02/10 12:16:30 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:16:30 ERROR : DJI_0708_20201128_003902.MP4: Failed to copy: can't upload files here
2021/02/10 12:16:30 ERROR : DJI_0708_20201128_003902_copy.MP4: Failed to copy: can't upload files here
2021/02/10 12:16:30 INFO  : There was nothing to transfer
2021/02/10 12:16:30 ERROR : Attempt 1/3 failed with 2 errors and: can't upload files here
2021/02/10 12:16:30 DEBUG : Google Photos path "album": List: dir=""
2021/02/10 12:16:30 DEBUG : Google Photos path "album": >List: err=<nil>
2021/02/10 12:16:30 DEBUG : Google Photos path "album": Waiting for checks to finish
2021/02/10 12:16:30 DEBUG : Google Photos path "album": Waiting for transfers to finish
2021/02/10 12:16:30 DEBUG : Google Photos path "album": Put: src=DJI_0708_20201128_003902.MP4
2021/02/10 12:16:30 DEBUG : DJI_0708_20201128_003902.MP4: Update: src=DJI_0708_20201128_003902.MP4
2021/02/10 12:16:30 DEBUG : Google Photos path "album": Put: src=DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:16:30 DEBUG : DJI_0708_20201128_003902.MP4: >Update: err=can't upload files here
2021/02/10 12:16:30 DEBUG : DJI_0708_20201128_003902_copy.MP4: Update: src=DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:16:30 DEBUG : DJI_0708_20201128_003902_copy.MP4: >Update: err=can't upload files here
2021/02/10 12:16:30 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:16:30 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:16:30 ERROR : DJI_0708_20201128_003902.MP4: Failed to copy: can't upload files here
2021/02/10 12:16:30 ERROR : DJI_0708_20201128_003902_copy.MP4: Failed to copy: can't upload files here
2021/02/10 12:16:30 INFO  : There was nothing to transfer
2021/02/10 12:16:30 ERROR : Attempt 2/3 failed with 2 errors and: can't upload files here
2021/02/10 12:16:30 DEBUG : Google Photos path "album": List: dir=""
2021/02/10 12:16:30 DEBUG : Google Photos path "album": >List: err=<nil>
2021/02/10 12:16:30 DEBUG : Google Photos path "album": Waiting for checks to finish
2021/02/10 12:16:30 DEBUG : Google Photos path "album": Waiting for transfers to finish
2021/02/10 12:16:30 DEBUG : Google Photos path "album": Put: src=DJI_0708_20201128_003902.MP4
2021/02/10 12:16:30 DEBUG : DJI_0708_20201128_003902.MP4: Update: src=DJI_0708_20201128_003902.MP4
2021/02/10 12:16:30 DEBUG : Google Photos path "album": Put: src=DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:16:30 DEBUG : DJI_0708_20201128_003902_copy.MP4: Update: src=DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:16:30 DEBUG : DJI_0708_20201128_003902_copy.MP4: >Update: err=can't upload files here
2021/02/10 12:16:30 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:16:30 DEBUG : DJI_0708_20201128_003902.MP4: >Update: err=can't upload files here
2021/02/10 12:16:30 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:16:30 ERROR : DJI_0708_20201128_003902_copy.MP4: Failed to copy: can't upload files here
2021/02/10 12:16:30 ERROR : DJI_0708_20201128_003902.MP4: Failed to copy: can't upload files here
2021/02/10 12:16:30 INFO  : There was nothing to transfer
2021/02/10 12:16:30 ERROR : Attempt 3/3 failed with 2 errors and: can't upload files here
2021/02/10 12:16:30 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 2 (retrying may help)
Elapsed time:         3.8s

2021/02/10 12:16:30 DEBUG : 6 go routines active
2021/02/10 12:16:30 Failed to copy with 2 errors: last error was: can't upload files here
2021/02/10 12:18:19 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "-vv" "--log-file=/Users/m/code/projects/mediasort/VIDEOS/gphotos_log.txt" "copy" "/Users/m/code/projects/mediasort/VIDEOS/TESTING/" "gphotos:album"]
2021/02/10 12:18:19 DEBUG : Creating backend with remote "/Users/m/code/projects/mediasort/VIDEOS/TESTING/"
2021/02/10 12:18:19 DEBUG : Using config file from "/Users/m/.config/rclone/rclone.conf"
2021/02/10 12:18:19 DEBUG : Creating backend with remote "gphotos:album"
2021/02/10 12:18:19 DEBUG : Google Photos path "album": List: dir=""
2021/02/10 12:18:22 DEBUG : Google Photos path "album": >List: err=<nil>
2021/02/10 12:18:22 DEBUG : Google Photos path "album": Put: src=.DS_Store
2021/02/10 12:18:22 DEBUG : .DS_Store: Update: src=.DS_Store
2021/02/10 12:18:22 DEBUG : .DS_Store: >Update: err=can't upload files here
2021/02/10 12:18:22 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:18:22 DEBUG : Google Photos path "album": Put: src=DJI/DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:18:22 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Update: src=DJI/DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:18:22 ERROR : .DS_Store: Failed to copy: can't upload files here
2021/02/10 12:18:22 DEBUG : Google Photos path "album": Put: src=DJI/DJI_0708_20201128_003902.MP4
2021/02/10 12:18:22 DEBUG : DJI/DJI_0708_20201128_003902.MP4: Update: src=DJI/DJI_0708_20201128_003902.MP4
2021/02/10 12:18:22 DEBUG : Google Photos path "album": Waiting for checks to finish
2021/02/10 12:18:22 DEBUG : Google Photos path "album": Waiting for transfers to finish
2021/02/10 12:19:19 INFO  : 
Transferred:   	   36.476M / 96.201 MBytes, 38%, 657.672 kBytes/s, ETA 1m32s
Errors:                 1 (retrying may help)
Transferred:            0 / 2, 0%
Elapsed time:       1m0.0s
Transferring:
 *              DJI/DJI_0708_20201128_003902.MP4: 38% /48.100M, 335.478k/s, 1m30s
 *         DJI/DJI_0708_20201128_003902_copy.MP4: 37% /48.100M, 335.831k/s, 1m31s

2021/02/10 12:20:19 INFO  : 
Transferred:   	   75.648M / 96.201 MBytes, 79%, 663.226 kBytes/s, ETA 31s
Errors:                 1 (retrying may help)
Transferred:            0 / 2, 0%
Elapsed time:       2m0.0s
Transferring:
 *              DJI/DJI_0708_20201128_003902.MP4: 79% /48.100M, 326.635k/s, 31s
 *         DJI/DJI_0708_20201128_003902_copy.MP4: 77% /48.100M, 324.583k/s, 33s

2021/02/10 12:21:03 DEBUG : DJI/DJI_0708_20201128_003902.MP4: >Update: err=<nil>
2021/02/10 12:21:03 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:21:03 DEBUG : DJI/DJI_0708_20201128_003902.MP4: Size: 
2021/02/10 12:21:03 DEBUG : DJI/DJI_0708_20201128_003902.MP4: >Size: 
2021/02/10 12:21:03 INFO  : DJI/DJI_0708_20201128_003902.MP4: Copied (new)
2021/02/10 12:21:05 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Update: err=<nil>
2021/02/10 12:21:05 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:21:05 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Size: 
2021/02/10 12:21:05 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Size: 
2021/02/10 12:21:05 INFO  : DJI/DJI_0708_20201128_003902_copy.MP4: Copied (new)
2021/02/10 12:21:05 ERROR : Attempt 1/3 failed with 1 errors and: can't upload files here
2021/02/10 12:21:05 DEBUG : Google Photos path "album": List: dir=""
2021/02/10 12:21:05 DEBUG : Google Photos path "album": Error reading media count: strconv.ParseInt: parsing "": invalid syntax
2021/02/10 12:21:05 DEBUG : Google Photos path "album": >List: err=<nil>
2021/02/10 12:21:05 DEBUG : Google Photos path "album": List: dir="DJI"
2021/02/10 12:21:05 DEBUG : Google Photos path "album": Put: src=.DS_Store
2021/02/10 12:21:05 DEBUG : .DS_Store: Update: src=.DS_Store
2021/02/10 12:21:05 DEBUG : .DS_Store: >Update: err=can't upload files here
2021/02/10 12:21:05 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:21:05 ERROR : .DS_Store: Failed to copy: can't upload files here
2021/02/10 12:21:06 DEBUG : Google Photos path "album": >List: err=<nil>
2021/02/10 12:21:06 DEBUG : DJI/DJI_0708_20201128_003902.MP4: Size: 
2021/02/10 12:21:06 DEBUG : DJI/DJI_0708_20201128_003902.MP4: >Size: 
2021/02/10 12:21:06 DEBUG : Google Photos path "album": Put: src=DJI/DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:21:06 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Update: src=DJI/DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:21:06 DEBUG : DJI/DJI_0708_20201128_003902.MP4: Size: 
2021/02/10 12:21:06 DEBUG : DJI/DJI_0708_20201128_003902.MP4: >Size: 
2021/02/10 12:21:06 DEBUG : DJI/DJI_0708_20201128_003902.MP4: Sizes identical
2021/02/10 12:21:06 DEBUG : DJI/DJI_0708_20201128_003902.MP4: Unchanged skipping
2021/02/10 12:21:06 DEBUG : Google Photos path "album": Waiting for checks to finish
2021/02/10 12:21:06 DEBUG : Google Photos path "album": Waiting for transfers to finish
2021/02/10 12:21:19 INFO  : 
Transferred:   	  104.759M / 144.301 MBytes, 73%, 609.580 kBytes/s, ETA 1m6s
Errors:                 1 (retrying may help)
Checks:                 1 / 1, 100%
Transferred:            2 / 3, 67%
Elapsed time:       3m0.0s
Transferring:
 *         DJI/DJI_0708_20201128_003902_copy.MP4: 17% /48.100M, 698.229k/s, 57s

2021/02/10 12:22:19 INFO  : 
Transferred:   	  144.150M / 144.301 MBytes, 100%, 625.514 kBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:                 1 / 1, 100%
Transferred:            2 / 3, 67%
Elapsed time:       4m0.0s
Transferring:
 *         DJI/DJI_0708_20201128_003902_copy.MP4: 99% /48.100M, 662.104k/s, 0s

2021/02/10 12:22:23 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Update: err=<nil>
2021/02/10 12:22:23 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:22:23 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Size: 
2021/02/10 12:22:23 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Size: 
2021/02/10 12:22:23 INFO  : DJI/DJI_0708_20201128_003902_copy.MP4: Copied (new)
2021/02/10 12:22:23 ERROR : Attempt 2/3 failed with 1 errors and: can't upload files here
2021/02/10 12:22:23 DEBUG : Google Photos path "album": List: dir=""
2021/02/10 12:22:23 DEBUG : Google Photos path "album": Error reading media count: strconv.ParseInt: parsing "": invalid syntax
2021/02/10 12:22:23 DEBUG : Google Photos path "album": >List: err=<nil>
2021/02/10 12:22:23 DEBUG : Google Photos path "album": List: dir="DJI"
2021/02/10 12:22:23 DEBUG : Google Photos path "album": Put: src=.DS_Store
2021/02/10 12:22:23 DEBUG : .DS_Store: Update: src=.DS_Store
2021/02/10 12:22:23 DEBUG : .DS_Store: >Update: err=can't upload files here
2021/02/10 12:22:23 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:22:23 ERROR : .DS_Store: Failed to copy: can't upload files here
2021/02/10 12:22:23 DEBUG : Google Photos path "album": >List: err=<nil>
2021/02/10 12:22:23 DEBUG : DJI/DJI_0708_20201128_003902.MP4: Size: 
2021/02/10 12:22:23 DEBUG : DJI/DJI_0708_20201128_003902.MP4: >Size: 
2021/02/10 12:22:23 DEBUG : Google Photos path "album": Put: src=DJI/DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:22:23 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Update: src=DJI/DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:22:23 DEBUG : DJI/DJI_0708_20201128_003902.MP4: Size: 
2021/02/10 12:22:23 DEBUG : DJI/DJI_0708_20201128_003902.MP4: >Size: 
2021/02/10 12:22:23 DEBUG : DJI/DJI_0708_20201128_003902.MP4: Sizes identical
2021/02/10 12:22:23 DEBUG : DJI/DJI_0708_20201128_003902.MP4: Unchanged skipping
2021/02/10 12:22:23 DEBUG : Google Photos path "album": Waiting for checks to finish
2021/02/10 12:22:23 DEBUG : Google Photos path "album": Waiting for transfers to finish
2021/02/10 12:23:19 INFO  : 
Transferred:   	  181.844M / 192.402 MBytes, 95%, 630.435 kBytes/s, ETA 17s
Errors:                 1 (retrying may help)
Checks:                 2 / 2, 100%
Transferred:            3 / 4, 75%
Elapsed time:       5m0.0s
Transferring:
 *         DJI/DJI_0708_20201128_003902_copy.MP4: 78% /48.100M, 696.641k/s, 15s

2021/02/10 12:23:37 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Update: err=<nil>
2021/02/10 12:23:37 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:23:37 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Size: 
2021/02/10 12:23:37 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Size: 
2021/02/10 12:23:37 INFO  : DJI/DJI_0708_20201128_003902_copy.MP4: Copied (new)
2021/02/10 12:23:37 ERROR : Attempt 3/3 failed with 1 errors and: can't upload files here
2021/02/10 12:23:37 INFO  : 
Transferred:   	  192.402M / 192.402 MBytes, 100%, 627.391 kBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:                 2 / 2, 100%
Transferred:            4 / 4, 100%
Elapsed time:      5m18.7s

2021/02/10 12:23:37 DEBUG : 7 go routines active
2021/02/10 12:23:37 Failed to copy: can't upload files here
2021/02/10 12:25:21 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "-vv" "--log-file=/Users/m/code/projects/mediasort/VIDEOS/gphotos_log.txt" "copy" "/Users/m/code/projects/mediasort/VIDEOS/TESTING/" "gphotos:album"]
2021/02/10 12:25:21 DEBUG : Creating backend with remote "/Users/m/code/projects/mediasort/VIDEOS/TESTING/"
2021/02/10 12:25:21 DEBUG : Using config file from "/Users/m/.config/rclone/rclone.conf"
2021/02/10 12:25:21 DEBUG : Creating backend with remote "gphotos:album"
2021/02/10 12:25:21 DEBUG : Google Photos path "album": List: dir=""
2021/02/10 12:25:24 DEBUG : Google Photos path "album": Error reading media count: strconv.ParseInt: parsing "": invalid syntax
2021/02/10 12:25:24 DEBUG : Google Photos path "album": >List: err=<nil>
2021/02/10 12:25:24 DEBUG : Google Photos path "album": List: dir="DJI"
2021/02/10 12:25:24 DEBUG : Google Photos path "album": Put: src=.DS_Store
2021/02/10 12:25:24 DEBUG : .DS_Store: Update: src=.DS_Store
2021/02/10 12:25:24 DEBUG : .DS_Store: >Update: err=can't upload files here
2021/02/10 12:25:24 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:25:24 ERROR : .DS_Store: Failed to copy: can't upload files here
2021/02/10 12:25:24 DEBUG : Google Photos path "album": >List: err=<nil>
2021/02/10 12:25:24 DEBUG : Google Photos path "album": Put: src=DJI/.DS_Store
2021/02/10 12:25:24 DEBUG : Google Photos path "album": Waiting for checks to finish
2021/02/10 12:25:24 DEBUG : Google Photos path "album": Waiting for transfers to finish
2021/02/10 12:25:24 DEBUG : Google Photos path "album": Put: src=DJI/DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:25:24 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Update: src=DJI/DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:25:24 DEBUG : DJI/.DS_Store: Update: src=DJI/.DS_Store
2021/02/10 12:25:28 DEBUG : DJI/.DS_Store: >Update: err=upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:25:28 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:25:28 ERROR : DJI/.DS_Store: Failed to copy: upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:26:20 INFO  : 
Transferred:   	   34.893M / 48.106 MBytes, 73%, 636.800 kBytes/s, ETA 21s
Errors:                 2 (retrying may help)
Transferred:            0 / 1, 0%
Elapsed time:       1m0.3s
Transferring:
 *         DJI/DJI_0708_20201128_003902_copy.MP4: 72% /48.100M, 680.386k/s, 19s

2021/02/10 12:26:42 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Update: err=<nil>
2021/02/10 12:26:42 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:26:42 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Size: 
2021/02/10 12:26:42 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Size: 
2021/02/10 12:26:42 INFO  : DJI/DJI_0708_20201128_003902_copy.MP4: Copied (new)
2021/02/10 12:26:42 ERROR : Attempt 1/3 failed with 2 errors and: upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:26:42 DEBUG : Google Photos path "album": List: dir=""
2021/02/10 12:26:42 DEBUG : Google Photos path "album": Error reading media count: strconv.ParseInt: parsing "": invalid syntax
2021/02/10 12:26:42 DEBUG : Google Photos path "album": >List: err=<nil>
2021/02/10 12:26:42 DEBUG : Google Photos path "album": List: dir="DJI"
2021/02/10 12:26:42 DEBUG : Google Photos path "album": Put: src=.DS_Store
2021/02/10 12:26:42 DEBUG : .DS_Store: Update: src=.DS_Store
2021/02/10 12:26:42 DEBUG : .DS_Store: >Update: err=can't upload files here
2021/02/10 12:26:42 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:26:42 ERROR : .DS_Store: Failed to copy: can't upload files here
2021/02/10 12:26:43 DEBUG : Google Photos path "album": >List: err=<nil>
2021/02/10 12:26:43 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Size: 
2021/02/10 12:26:43 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Size: 
2021/02/10 12:26:43 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Size: 
2021/02/10 12:26:43 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Size: 
2021/02/10 12:26:43 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Sizes identical
2021/02/10 12:26:43 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Unchanged skipping
2021/02/10 12:26:43 DEBUG : Google Photos path "album": Waiting for checks to finish
2021/02/10 12:26:43 DEBUG : Google Photos path "album": Waiting for transfers to finish
2021/02/10 12:26:43 DEBUG : Google Photos path "album": Put: src=DJI/.DS_Store
2021/02/10 12:26:43 DEBUG : DJI/.DS_Store: Update: src=DJI/.DS_Store
2021/02/10 12:26:44 DEBUG : DJI/.DS_Store: >Update: err=upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:26:44 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:26:44 ERROR : DJI/.DS_Store: Failed to copy: upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:26:44 ERROR : Attempt 2/3 failed with 2 errors and: upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:26:44 DEBUG : Google Photos path "album": List: dir=""
2021/02/10 12:26:44 DEBUG : Google Photos path "album": Error reading media count: strconv.ParseInt: parsing "": invalid syntax
2021/02/10 12:26:44 DEBUG : Google Photos path "album": >List: err=<nil>
2021/02/10 12:26:44 DEBUG : Google Photos path "album": List: dir="DJI"
2021/02/10 12:26:44 DEBUG : Google Photos path "album": Put: src=.DS_Store
2021/02/10 12:26:44 DEBUG : .DS_Store: Update: src=.DS_Store
2021/02/10 12:26:44 DEBUG : .DS_Store: >Update: err=can't upload files here
2021/02/10 12:26:44 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:26:44 ERROR : .DS_Store: Failed to copy: can't upload files here
2021/02/10 12:26:45 DEBUG : Google Photos path "album": >List: err=<nil>
2021/02/10 12:26:45 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Size: 
2021/02/10 12:26:45 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Size: 
2021/02/10 12:26:45 DEBUG : Google Photos path "album": Waiting for checks to finish
2021/02/10 12:26:45 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Size: 
2021/02/10 12:26:45 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Size: 
2021/02/10 12:26:45 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Sizes identical
2021/02/10 12:26:45 DEBUG : Google Photos path "album": Put: src=DJI/.DS_Store
2021/02/10 12:26:45 DEBUG : DJI/.DS_Store: Update: src=DJI/.DS_Store
2021/02/10 12:26:45 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Unchanged skipping
2021/02/10 12:26:45 DEBUG : Google Photos path "album": Waiting for transfers to finish
2021/02/10 12:26:46 DEBUG : DJI/.DS_Store: >Update: err=upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:26:46 DEBUG : Google Photos path "album": >Put: 
2021/02/10 12:26:46 ERROR : DJI/.DS_Store: Failed to copy: upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:26:46 ERROR : Attempt 3/3 failed with 2 errors and: upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:26:46 INFO  : 
Transferred:   	   48.118M / 48.118 MBytes, 100%, 611.219 kBytes/s, ETA 0s
Errors:                 2 (retrying may help)
Checks:                 2 / 2, 100%
Transferred:            1 / 1, 100%
Elapsed time:      1m26.0s

2021/02/10 12:26:46 DEBUG : 5 go routines active
2021/02/10 12:26:46 Failed to copy with 2 errors: last error was: upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:47:14 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "-vv" "--log-file=/Users/m/code/projects/mediasort/VIDEOS/gphotos_log.txt" "copy" "/Users/m/code/projects/mediasort/VIDEOS/TESTING/" "gphotos:upload"]
2021/02/10 12:47:14 DEBUG : Creating backend with remote "/Users/m/code/projects/mediasort/VIDEOS/TESTING/"
2021/02/10 12:47:14 DEBUG : Using config file from "/Users/m/.config/rclone/rclone.conf"
2021/02/10 12:47:14 DEBUG : Creating backend with remote "gphotos:upload"
2021/02/10 12:47:14 DEBUG : Google Photos path "upload": List: dir=""
2021/02/10 12:47:14 DEBUG : Google Photos path "upload": >List: err=directory not found
2021/02/10 12:47:14 DEBUG : Google Photos path "upload": Put: src=.DS_Store
2021/02/10 12:47:14 DEBUG : .DS_Store: Update: src=.DS_Store
2021/02/10 12:47:14 DEBUG : Google Photos path "upload": Put: src=DJI/.DS_Store
2021/02/10 12:47:14 DEBUG : DJI/.DS_Store: Update: src=DJI/.DS_Store
2021/02/10 12:47:14 DEBUG : Google Photos path "upload": Waiting for checks to finish
2021/02/10 12:47:14 DEBUG : Google Photos path "upload": Waiting for transfers to finish
2021/02/10 12:47:14 DEBUG : Google Photos path "upload": Put: src=DJI/DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:47:14 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Update: src=DJI/DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:47:17 DEBUG : DJI/.DS_Store: >Update: err=upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:47:17 DEBUG : Google Photos path "upload": >Put: 
2021/02/10 12:47:17 ERROR : DJI/.DS_Store: Failed to copy: upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:47:17 DEBUG : .DS_Store: >Update: err=upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:47:17 DEBUG : Google Photos path "upload": >Put: 
2021/02/10 12:47:17 ERROR : .DS_Store: Failed to copy: upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:48:14 INFO  : 
Transferred:   	   40.164M / 48.112 MBytes, 83%, 685.544 kBytes/s, ETA 11s
Errors:                 2 (retrying may help)
Transferred:            0 / 1, 0%
Elapsed time:       1m0.3s
Transferring:
 *         DJI/DJI_0708_20201128_003902_copy.MP4: 83% /48.100M, 688.533k/s, 11s

2021/02/10 12:48:29 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Update: err=<nil>
2021/02/10 12:48:29 DEBUG : Google Photos path "upload": >Put: 
2021/02/10 12:48:29 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Size: 
2021/02/10 12:48:29 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Size: 
2021/02/10 12:48:29 INFO  : DJI/DJI_0708_20201128_003902_copy.MP4: Copied (new)
2021/02/10 12:48:29 ERROR : Attempt 1/3 failed with 2 errors and: upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:48:29 DEBUG : Google Photos path "upload": List: dir=""
2021/02/10 12:48:29 DEBUG : Google Photos path "upload": >List: err=directory not found
2021/02/10 12:48:29 DEBUG : Google Photos path "upload": Put: src=.DS_Store
2021/02/10 12:48:29 DEBUG : .DS_Store: Update: src=.DS_Store
2021/02/10 12:48:29 DEBUG : Google Photos path "upload": Put: src=DJI/DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:48:29 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Update: src=DJI/DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:48:29 DEBUG : Google Photos path "upload": Put: src=DJI/.DS_Store
2021/02/10 12:48:29 DEBUG : DJI/.DS_Store: Update: src=DJI/.DS_Store
2021/02/10 12:48:29 DEBUG : Google Photos path "upload": Waiting for checks to finish
2021/02/10 12:48:29 DEBUG : Google Photos path "upload": Waiting for transfers to finish
2021/02/10 12:48:30 DEBUG : .DS_Store: >Update: err=upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:48:30 DEBUG : Google Photos path "upload": >Put: 
2021/02/10 12:48:30 ERROR : .DS_Store: Failed to copy: upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:48:30 DEBUG : DJI/.DS_Store: >Update: err=upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:48:30 DEBUG : Google Photos path "upload": >Put: 
2021/02/10 12:48:30 ERROR : DJI/.DS_Store: Failed to copy: upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:49:14 INFO  : 
Transferred:   	   78.526M / 96.224 MBytes, 82%, 670.140 kBytes/s, ETA 27s
Errors:                 2 (retrying may help)
Transferred:            1 / 2, 50%
Elapsed time:       2m0.3s
Transferring:
 *         DJI/DJI_0708_20201128_003902_copy.MP4: 63% /48.100M, 681.421k/s, 26s

2021/02/10 12:49:44 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Update: err=<nil>
2021/02/10 12:49:44 DEBUG : Google Photos path "upload": >Put: 
2021/02/10 12:49:44 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Size: 
2021/02/10 12:49:44 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Size: 
2021/02/10 12:49:44 INFO  : DJI/DJI_0708_20201128_003902_copy.MP4: Copied (new)
2021/02/10 12:49:44 ERROR : Attempt 2/3 failed with 2 errors and: upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:49:44 DEBUG : Google Photos path "upload": List: dir=""
2021/02/10 12:49:44 DEBUG : Google Photos path "upload": >List: err=directory not found
2021/02/10 12:49:44 DEBUG : Google Photos path "upload": Put: src=.DS_Store
2021/02/10 12:49:44 DEBUG : .DS_Store: Update: src=.DS_Store
2021/02/10 12:49:44 DEBUG : Google Photos path "upload": Put: src=DJI/DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:49:44 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Update: src=DJI/DJI_0708_20201128_003902_copy.MP4
2021/02/10 12:49:44 DEBUG : Google Photos path "upload": Put: src=DJI/.DS_Store
2021/02/10 12:49:44 DEBUG : DJI/.DS_Store: Update: src=DJI/.DS_Store
2021/02/10 12:49:44 DEBUG : Google Photos path "upload": Waiting for checks to finish
2021/02/10 12:49:44 DEBUG : Google Photos path "upload": Waiting for transfers to finish
2021/02/10 12:49:45 DEBUG : .DS_Store: >Update: err=upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:49:45 DEBUG : Google Photos path "upload": >Put: 
2021/02/10 12:49:45 ERROR : .DS_Store: Failed to copy: upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:49:47 DEBUG : DJI/.DS_Store: >Update: err=upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:49:47 DEBUG : Google Photos path "upload": >Put: 
2021/02/10 12:49:47 ERROR : DJI/.DS_Store: Failed to copy: upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:50:14 INFO  : 
Transferred:   	  116.591M / 144.336 MBytes, 81%, 663.326 kBytes/s, ETA 42s
Errors:                 2 (retrying may help)
Transferred:            2 / 3, 67%
Elapsed time:       3m0.3s
Transferring:
 *         DJI/DJI_0708_20201128_003902_copy.MP4: 42% /48.100M, 693.919k/s, 40s

2021/02/10 12:51:08 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Update: err=<nil>
2021/02/10 12:51:08 DEBUG : Google Photos path "upload": >Put: 
2021/02/10 12:51:08 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: Size: 
2021/02/10 12:51:08 DEBUG : DJI/DJI_0708_20201128_003902_copy.MP4: >Size: 
2021/02/10 12:51:08 INFO  : DJI/DJI_0708_20201128_003902_copy.MP4: Copied (new)
2021/02/10 12:51:08 ERROR : Attempt 3/3 failed with 2 errors and: upload failed: Failed: There was an error while trying to create this media item. (3)
2021/02/10 12:51:08 INFO  : 
Transferred:   	  144.336M / 144.336 MBytes, 100%, 633.666 kBytes/s, ETA 0s
Errors:                 2 (retrying may help)
Transferred:            3 / 3, 100%
Elapsed time:      3m53.6s

2021/02/10 12:51:08 DEBUG : 5 go routines active
2021/02/10 12:51:08 Failed to copy with 2 errors: last error was: upload failed: Failed: There was an error while trying to create this media item. (3)

ncw · February 11, 2021, 4:55pm

I suspect the problem is with the API here. It is pretty dreadful and it isn't the one any of the Google products use (they have their own secret fully featured API as far as I can see).

Here are the docs for the upload

There is no way for rclone to set a date there

Will307 · February 23, 2021, 11:14am

Date taken should be set to Modified date if Date taken is empty. If rclone just simply did this small check everytime it downloads a video and update this one meta data field date taken it would correct this issue.

I take my Google content and upload to Amazon and this issue also exists in Amazon. Amazon sorts by date taken whether its an image or a video. An empty date taken just puts all sorts of videos into a 'no date' bucket.

ncw · February 23, 2021, 11:22am

Rclone would have to modify the media to do that wouldn't it?

I don't know if you could add an exif block with exiftool or something like that?

Will307 · February 23, 2021, 1:15pm

Yes, rclone would have to do that. I don't know what language rclone is written in but it would behave as an exif tool does for this one attribute. It would be the Media Create Date from what I can tell.

ExifTool Version Number : 11.88
File Name : IMG_4163.MOV
Directory : .
File Size : 23 MB
File Modification Date/Time : 2021:02:19 17:31:57-05:00
File Access Date/Time : 2021:02:23 05:53:59-05:00
File Inode Change Date/Time : 2021:02:19 13:35:29-05:00
File Permissions : rw-r--r--
File Type : MP4
File Type Extension : mp4
MIME Type : video/mp4
Major Brand : MP4 v2 [ISO 14496-14]
Minor Version : 0.0.0
Compatible Brands : isom, mp42
Movie Header Version : 0
Create Date : 0000:00:00 00:00:00
Modify Date : 0000:00:00 00:00:00
Time Scale : 30000
Duration : 0:00:38
Preferred Rate : 1
Preferred Volume : 100.00%
Preview Time : 0 s
Preview Duration : 0 s
Poster Time : 0 s
Selection Time : 0 s
Selection Duration : 0 s
Current Time : 0 s
Next Track ID : 3
Track Header Version : 0
Track Create Date : 0000:00:00 00:00:00
Track Modify Date : 0000:00:00 00:00:00
Track ID : 1
Track Duration : 0:00:38
Track Layer : 0
Track Volume : 0.00%
Image Width : 1920
Image Height : 1080
Compressor ID : avc1
Source Image Width : 1920
Source Image Height : 1080
X Resolution : 72
Y Resolution : 72
Bit Depth : 24
Video Frame Rate : 29.97
Graphics Mode : srcCopy
Op Color : 0 0 0
Matrix Structure : 1 0 0 0 1 0 0 0 1
Media Header Version : 0
Media Create Date : 0000:00:00 00:00:00
Media Modify Date : 0000:00:00 00:00:00
Media Time Scale : 44100
Media Duration : 0:00:38
Media Language Code : und
Handler Type : Audio Track
Handler Description : ISO Media file produced by Google Inc.
Audio Format : mp4a
Audio Channels : 2
Audio Bits Per Sample : 16
Audio Sample Rate : 44100
Balance : 0
Media Data Size : 24280488
Media Data Offset : 19580
Image Size : 1920x1080
Megapixels : 2.1
Avg Bitrate : 5.05 Mbps
Rotation : 0

ncw · February 23, 2021, 4:47pm

I think this is out of scope for rclone, alas.

It is written in Go.

Can you use a 3rd party tool to set the EXIF data on the file before you upload them with rclone? Then you've got the same files locally and uploaded.

Will307 · February 23, 2021, 8:25pm

Using a third party tool is possible. It would have to be one that can do conditional logic like 'if media created date is 'empty' write modified-date to media-created-date. Doing it all separate from rclone though just makes the whole process slower. Its like double the scans. rclone is the only place where it could do this right before updating the local file.

system · April 25, 2021, 4:25pm

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.