Rclone + Google Photos all photos zero bytes

LeoW · June 8, 2023, 5:05am

What is the problem you are having with rclone?

...
I created a config to mount Google Photos. That works. However, Windows file explorer can see the photos and screenshots on the drive, only they are all listed as being zero bytes and can't be opened.

If I launch the GUI and go to the "Explorer" menu, I can connect to the server, see the files, and download them as their proper, normal size, and they open.

When I run the mount command there are no errors.

...

Run the command 'rclone version' and share the full output of the command.

...

rclone v1.62.2

os/version: Microsoft Windows 11 Home 22H2 (64 bit)
os/kernel: 10.0.22621.1702 Build 22621.1702.1702 (x86_64)
os/type: windows
os/arch: amd64
go/version: go1.20.2
go/linking: static
go/tags: cmount

...

Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads
--> Yes

Which cloud storage system are you using? (eg Google Drive)

Google Photos (Google Drive Works on the same Gmail account)

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone --vfs-cache-mode full mount stugphoto:/ s:

The rclone config contents with secrets removed.


Select remote.
Choose a number from below, or type in an existing value.
 1 > ann
 2 > drew-google
 3 > drew-od
 4 > stug-google
 5 > stugphoto
 6 > surface7-sftp
remote> 5

Editing existing "stugphoto" remote with options:
- type: gphotos
- token: DELETED FROM OUTPUT

Option client_id.
OAuth Client Id.
Leave blank normally.
Enter a value. Press Enter to leave empty.
client_id>

Option client_secret.
OAuth Client Secret.
Leave blank normally.
Enter a value. Press Enter to leave empty.
client_secret>

Option read_only.
Set to make the Google Photos backend read only.
If you choose read only then rclone will only request read only access
to your photos, otherwise rclone will request full access.
Enter a boolean value (true or false). Press Enter for the default (false).
read_only>

Edit advanced config?
y) Yes
n) No (default)
y/n>

Already have a token - refresh?
y) Yes (default)
n) No
y/n>

Use web browser to automatically authenticate rclone with remote?
 * Say Y if the machine running rclone has a web browser you can use
 * Say N if running rclone on a (remote) machine without web browser access
If not sure try Y. If Y failed, try N.

y) Yes (default)
n) No
y/n>

2023/06/08 00:34:29 NOTICE: If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=OSfhFdfljbKM9tS6btWmBA
2023/06/08 00:34:29 NOTICE: Log in and authorize rclone for access
2023/06/08 00:34:29 NOTICE: Waiting for code...
2023/06/08 00:35:05 NOTICE: Got code
Warning

IMPORTANT: All media items uploaded to Google Photos with rclone
are stored in full resolution at original quality.  These uploads
will count towards storage in your Google Account.
y) Yes (default)
n) No
y/n>

Configuration complete.
Options:
- type: gphotos
- token: DELETED from config
Keep this "stugphoto" remote?
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d>

Current remotes:

Name                 Type
====                 ====
ann                  onedrive
drew-google          drive
drew-od              onedrive
stug-google          drive
stugphoto            gphotos
surface7-sftp        sftp

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q

A log from the command with the `-vv` flag

C:\Users\sarah>rclone -vv --vfs-cache-mode full mount stugphoto:/ s:
2023/06/08 00:52:01 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "-vv" "--vfs-cache-mode" "full" "mount" "stugphoto:/" "s:"]
2023/06/08 00:52:01 DEBUG : Creating backend with remote "stugphoto:/"
2023/06/08 00:52:01 DEBUG : Using config file from "C:\\Users\\sarah\\AppData\\Roaming\\rclone\\rclone.conf"
2023/06/08 00:52:01 DEBUG : fs cache: renaming cache item "stugphoto:/" to be canonical "stugphoto:"
2023/06/08 00:52:01 INFO  : Google Photos path "": poll-interval is not supported by this remote
2023/06/08 00:52:01 DEBUG : vfs cache: root is "C:\\Users\\sarah\\AppData\\Local\\rclone"
2023/06/08 00:52:01 DEBUG : vfs cache: data root is "\\\\?\\C:\\Users\\sarah\\AppData\\Local\\rclone\\vfs\\stugphoto"
2023/06/08 00:52:01 DEBUG : vfs cache: metadata root is "\\\\?\\C:\\Users\\sarah\\AppData\\Local\\rclone\\vfsMeta\\stugphoto"
2023/06/08 00:52:01 DEBUG : Creating backend with remote "C:/Users/sarah/AppData/Local/rclone/vfs/stugphoto/"
2023/06/08 00:52:01 DEBUG : fs cache: renaming cache item "C:/Users/sarah/AppData/Local/rclone/vfs/stugphoto/" to be canonical "//?/C:/Users/sarah/AppData/Local/rclone/vfs/stugphoto"
2023/06/08 00:52:01 DEBUG : Creating backend with remote "C:/Users/sarah/AppData/Local/rclone/vfsMeta/stugphoto/"
2023/06/08 00:52:01 DEBUG : fs cache: renaming cache item "C:/Users/sarah/AppData/Local/rclone/vfsMeta/stugphoto/" to be canonical "//?/C:/Users/sarah/AppData/Local/rclone/vfsMeta/stugphoto"
2023/06/08 00:52:01 DEBUG : Network mode mounting is disabled
2023/06/08 00:52:01 DEBUG : Mounting on "s:" ("stugphoto")
2023/06/08 00:52:01 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2023/06/08 00:52:01 DEBUG : Google Photos path "": Mounting with options: ["-o" "attr_timeout=1" "-o" "uid=-1" "-o" "gid=-1" "--FileSystemName=rclone" "-o" "volname=stugphoto"]
2023/06/08 00:52:01 DEBUG : Google Photos path "": Init:
2023/06/08 00:52:01 DEBUG : Google Photos path "": >Init:
2023/06/08 00:52:01 DEBUG : /: Statfs:
2023/06/08 00:52:01 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2023/06/08 00:52:01 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:01 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:01 DEBUG : /: Readlink:
2023/06/08 00:52:01 DEBUG : /: >Readlink: linkPath="", errc=-40
2023/06/08 00:52:01 DEBUG : /: Getxattr: name="non-existant-a11ec902d22f4ec49003af15282d3b00"
2023/06/08 00:52:01 DEBUG : /: >Getxattr: errc=-40, value=""
2023/06/08 00:52:01 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:01 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:01 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:01 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:01 DEBUG : /: Opendir:
2023/06/08 00:52:01 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/06/08 00:52:01 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/06/08 00:52:01 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/06/08 00:52:01 DEBUG : /: Statfs:
2023/06/08 00:52:01 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:01 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:01 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:01 DEBUG : /: >Getattr: errc=0
The service rclone has been started.
2023/06/08 00:52:01 DEBUG : /: Opendir:
2023/06/08 00:52:01 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2023/06/08 00:52:01 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/06/08 00:52:01 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/06/08 00:52:01 DEBUG : /: >Opendir: errc=0, fh=0x1
2023/06/08 00:52:01 DEBUG : /: Releasedir: fh=0x0
2023/06/08 00:52:01 DEBUG : /: >Releasedir: errc=0
2023/06/08 00:52:01 DEBUG : /: Releasedir: fh=0x1
2023/06/08 00:52:01 DEBUG : /: >Releasedir: errc=0
2023/06/08 00:52:01 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:01 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:01 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:01 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:01 DEBUG : /: Opendir:
2023/06/08 00:52:01 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/06/08 00:52:01 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/06/08 00:52:01 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/06/08 00:52:01 DEBUG : /: Releasedir: fh=0x0
2023/06/08 00:52:01 DEBUG : /: >Releasedir: errc=0
2023/06/08 00:52:05 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:05 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:05 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:05 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:05 DEBUG : /: Opendir:
2023/06/08 00:52:05 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/06/08 00:52:05 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/06/08 00:52:05 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/06/08 00:52:05 DEBUG : /: Releasedir: fh=0x0
2023/06/08 00:52:05 DEBUG : /: >Releasedir: errc=0
2023/06/08 00:52:05 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:05 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:05 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:05 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:05 DEBUG : /: Opendir:
2023/06/08 00:52:05 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/06/08 00:52:05 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/06/08 00:52:05 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/06/08 00:52:05 DEBUG : /: Releasedir: fh=0x0
2023/06/08 00:52:05 DEBUG : /: >Releasedir: errc=0
2023/06/08 00:52:05 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:05 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:05 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:05 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:05 DEBUG : /: Opendir:
2023/06/08 00:52:05 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/06/08 00:52:05 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/06/08 00:52:05 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/06/08 00:52:05 DEBUG : /: Statfs:
2023/06/08 00:52:05 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2023/06/08 00:52:05 DEBUG : /: Releasedir: fh=0x0
2023/06/08 00:52:05 DEBUG : /: >Releasedir: errc=0
2023/06/08 00:52:05 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:05 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:05 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:05 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:05 DEBUG : /: Opendir:
2023/06/08 00:52:05 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/06/08 00:52:05 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/06/08 00:52:05 DEBUG : /: >Opendir: errc=0, fh=0x0
2023/06/08 00:52:10 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:10 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:10 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:10 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:10 DEBUG : /: Opendir:
2023/06/08 00:52:10 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/06/08 00:52:10 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/06/08 00:52:10 DEBUG : /: >Opendir: errc=0, fh=0x1
2023/06/08 00:52:10 DEBUG : /: Releasedir: fh=0x1
2023/06/08 00:52:10 DEBUG : /: >Releasedir: errc=0
2023/06/08 00:52:10 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:10 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:10 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:10 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:10 DEBUG : /: Opendir:
2023/06/08 00:52:10 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/06/08 00:52:10 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/06/08 00:52:10 DEBUG : /: >Opendir: errc=0, fh=0x1
2023/06/08 00:52:10 DEBUG : /: Releasedir: fh=0x1
2023/06/08 00:52:10 DEBUG : /: >Releasedir: errc=0
2023/06/08 00:52:13 DEBUG : /autorun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:13 DEBUG : Google Photos path "": List: dir=""
2023/06/08 00:52:13 DEBUG : Google Photos path "": >List: err=<nil>
2023/06/08 00:52:13 DEBUG : /autorun.inf: >Getattr: errc=-2
2023/06/08 00:52:13 DEBUG : /autorun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:13 DEBUG : /autorun.inf: >Getattr: errc=-2
2023/06/08 00:52:13 DEBUG : /autorun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:13 DEBUG : /autorun.inf: >Getattr: errc=-2
2023/06/08 00:52:13 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:13 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:13 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:13 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:13 DEBUG : /: Opendir:
2023/06/08 00:52:13 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2023/06/08 00:52:13 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2023/06/08 00:52:13 DEBUG : /: >Opendir: errc=0, fh=0x1
2023/06/08 00:52:13 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:13 DEBUG : /: >Getattr: errc=0
2023/06/08 00:52:13 DEBUG : /: Releasedir: fh=0x1
2023/06/08 00:52:13 DEBUG : /: >Releasedir: errc=0
2023/06/08 00:52:13 DEBUG : /AutoRun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/06/08 00:52:13 DEBUG : /AutoRun.inf: >Getattr: errc=-2
2023/06/08 00:53:01 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2023/06/08 00:54:01 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)

kapitainsky · June 8, 2023, 7:00am

This is all explained in the docs

The Google Photos API does not return the size of media. This means that when syncing to Google Photos, rclone can only do a file existence check.

It is possible to read the size of the media, but this needs an extra HTTP HEAD request per media item so is very slow and uses up a lot of transactions. This can be enabled with the --gphotos-read-size option or the read_size = true config parameter.

If you want to use the backend with rclone mount you will need to enable this flag otherwise you will not be able to read media off the mount.

LeoW · June 8, 2023, 2:06pm

This option is PERFECT!

I can both see the file size and open the files.
They're only going to be jpeg picture files and Screenshots and all I'll want to do is copy the most recent photos down to my computer without having to log into a browser and go to photos.Google.com.

Thanks again!

LeoW · June 8, 2023, 2:36pm

I never would have found that switch option on the command line by myself even with the documents so thank you again.

LeoW · June 8, 2023, 3:07pm

Another follow question:
How/where do I add the -gphotos-read-size when mounting via the gui interface. I didnt see an option to enable this flag.

Thanks..

kapitainsky · June 8, 2023, 3:08pm

Sorry but what do you mean by GUI interface? Can you post picture?

LeoW · June 8, 2023, 3:30pm

I'm referring to the gui that comes up when you type:
rclone rcd --rc-web-gui

There is an option to mount one of your remotes. When you select that option a box pops up with a menu of all kinds of choices where you can choose to turn on caching for example but there is no option to enable the photo size option flag.

When mounting via the new GUI interface is where I am referring to about putting in the:

rclone rcd --rc-web-gui

--gphotos-read-size flag.

kapitainsky · June 8, 2023, 3:55pm

OK. there is nowhere to.

It is possible to read the size of the media, but this needs an extra HTTP HEAD request per media item so is very slow and uses up a lot of transactions. This can be enabled with the --gphotos-read-size option or the read_size = true config parameter.

you can put instead read_size = true in the config of your remote in the config file

kapitainsky · June 8, 2023, 4:18pm

unless bottom left there is field for ExtraFlags - you can try it

LeoW · June 8, 2023, 7:30pm

This Worked!

I edited the config file and put this command in and now I can see the size and open the files.

LeoW · June 8, 2023, 7:34pm

Tried adding to the empty field on the left side of GUI but it just caused more errors.

Taking lots of notes cause this stuff is confusing for someone new.

Can't just give people the website address and tell them to go use it. must also send the other person that you want to have use it the software a list of the common commands and flags and edits to the config file and the command for entering the GUI et cetera.

Doesn't seem like the GUI is going to work for setting up the Google photo option it's more reliable or only reliable at the command prompt.

kapitainsky · June 8, 2023, 7:49pm

I am sorry but I know nothing about this GUI - but creating another remote with required flags sounds for me like universal solution.

system · July 8, 2023, 7:49pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.