Fresh Raspberry Pi build - Google Drive 404 Error - Failed to ls: googleapi: Error 404: File not found

bullishpip · November 6, 2019, 3:02pm

What is the problem you are having with rclone?

Failed to ls: googleapi: Error 404: File not found:

What is your rclone version (output from `rclone version`)

root@teslausb:~# rclone --version

rclone v1.49.5

os/arch: linux/arm
go version: go1.12.10

and rclone 1.50.1

Which OS you are using and how many bits (eg Windows 7, 64 bit)

armv6l GNU/Linux

Raspbian Buster Lite
Minimal image based on Debian Buster
Version: September 2019

Release date: 2019-09-26
Kernel version: 4.19

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone ls

A log from the command with the `-vv` flag (eg output from `rclone -vv copy /tmp remote:tmp`)

root@teslausb:~# rclone -vv ls gdrive:
2019/11/06 15:00:00 DEBUG : rclone: Version "v1.49.5" starting with parameters ["rclone" "-vv" "ls" "gdrive:"]
2019/11/06 15:00:00 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2019/11/06 15:00:02 Failed to ls: googleapi: Error 404: File not found: 0ABUrlFLwJwDzUk9PVA., notFound

Rclone config

root@teslausb:~# rclone config show
[gdrive]
type = drive
scope = drive.file
token = {"access_token"

Context

This same setup was working fine just a couple of days ago.

I've rebuilt and reinstalled several different versions of the Raspberry Pi and rclone all resulting in the same error message.

Questions

Is anyone else having this issue?

Would this be a Google Drive issue?

Thoughts?

Animosity022 · November 6, 2019, 3:05pm

(Sorry I see it now)

Your scope is wrong:
scope = drive.file

should be

scope = drive

bullishpip · November 6, 2019, 3:08pm

I have been using the scope drive.file without issue for more than a month now.

The scope is intentional -- I only want the Raspberry Pi to be able to read/write to files rclone created, no other files.

Animosity022 · November 6, 2019, 3:09pm

Did it just break then after working for a month?

Can you test it without the drive.file and see if you get files?

bullishpip · November 6, 2019, 3:10pm

Correct. Just broke within the last day or two.

Animosity022 · November 6, 2019, 3:11pm

Is that your entire rclone.conf as well?

Can you test without the scope and see if you get results?

bullishpip · November 6, 2019, 3:16pm

Same problem with full scope opened up

root@teslausb:~# rclone -vv ls gdrive:
2019/11/06 15:13:11 DEBUG : rclone: Version "v1.49.5" starting with parameters ["rclone" "-vv" "ls" "gdrive:"]
2019/11/06 15:13:11 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2019/11/06 15:13:12 Failed to ls: googleapi: Error 404: File not found: 0ABUrlFLwJwDzUk9PVA., notFound
root@teslausb:~# cat /root/.config/rclone/rclone.conf
[gdrive]
type = drive
scope = drive
token = {"access_token":"...}

root@teslausb:~#

Animosity022 · November 6, 2019, 3:18pm

Just for sanity as I'm using 1.50.1. Can you use the latest as well?

Can you run on that and run the same command with --dump headers and share the output.

bullishpip · November 6, 2019, 3:26pm

root@teslausb:~# rclone config show
[gdrive]
type = drive
scope = drive.file
token = {"access_token":...}

root@teslausb:~# rclone -vv ls gdrive: --dump headers
2019/11/06 15:24:20 DEBUG : rclone: Version "v1.50.1" starting with parameters ["rclone" "-vv" "ls" "gdrive:" "--dump" "headers"]
2019/11/06 15:24:20 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2019/11/06 15:24:20 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2019/11/06 15:24:20 DEBUG : HTTP REQUEST (req 0x3b9a500)
2019/11/06 15:24:20 DEBUG : GET /drive/v3/files/root?alt=json&fields=id&prettyPrint=false&supportsAllDrives=true HTTP/1.1
Host: www.googleapis.com
User-Agent: rclone/v1.50.1
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.11.0 gdcl/20190802
Accept-Encoding: gzip
2019/11/06 15:24:20 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2019/11/06 15:24:22 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2019/11/06 15:24:22 DEBUG : HTTP RESPONSE (req 0x3b9a500)
2019/11/06 15:24:22 DEBUG : HTTP/1.1 404 Not Found
Transfer-Encoding: chunked
Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000
Cache-Control: private, max-age=0
Content-Type: application/json; charset=UTF-8
Date: Wed, 06 Nov 2019 15:24:22 GMT
Expires: Wed, 06 Nov 2019 15:24:22 GMT
Server: GSE
Vary: Origin
Vary: X-Origin
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block
2019/11/06 15:24:22 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2019/11/06 15:24:22 Failed to create file system for "gdrive:": couldn't find root directory ID: googleapi: Error 404: File not found: 0ABUrlFLwJwDzUk9PVA., notFound
root@teslausb:~#

Animosity022 · November 6, 2019, 3:37pm

Can you try to authorize the account again? The first URL seems off as it doesn't have the token in it.

bullishpip · November 6, 2019, 3:44pm

Token refreshed... same issue...

root@teslausb:~# rclone -vv ls gdrive: --dump headers
2019/11/06 15:42:29 DEBUG : rclone: Version "v1.50.1" starting with parameters ["rclone" "-vv" "ls" "gdrive:" "--dump" "headers"]
2019/11/06 15:42:29 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2019/11/06 15:42:29 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2019/11/06 15:42:29 DEBUG : HTTP REQUEST (req 0x2392500)
2019/11/06 15:42:29 DEBUG : GET /drive/v3/files/root?alt=json&fields=id&prettyPrint=false&supportsAllDrives=true HTTP/1.1
Host: www.googleapis.com
User-Agent: rclone/v1.50.1
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.11.0 gdcl/20190802
Accept-Encoding: gzip

2019/11/06 15:42:29 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2019/11/06 15:42:31 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2019/11/06 15:42:31 DEBUG : HTTP RESPONSE (req 0x2392500)
2019/11/06 15:42:31 DEBUG : HTTP/1.1 404 Not Found
Transfer-Encoding: chunked
Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000
Cache-Control: private, max-age=0
Content-Type: application/json; charset=UTF-8
Date: Wed, 06 Nov 2019 15:42:31 GMT
Expires: Wed, 06 Nov 2019 15:42:31 GMT
Server: GSE
Vary: Origin
Vary: X-Origin
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block

2019/11/06 15:42:31 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2019/11/06 15:42:31 Failed to create file system for "gdrive:": couldn't find root directory ID: googleapi: Error 404: File not found: 0ABUrlFLwJwDzUk9PVA., notFound

Animosity022 · November 6, 2019, 3:49pm

I'm sorry as I fixed my file as I get the same first URL as you are seeing as I had a root folder setup.

2019/11/06 10:47:48 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2019/11/06 10:47:48 DEBUG : TEST: Saved new token in config file
2019/11/06 10:47:48 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2019/11/06 10:47:48 DEBUG : HTTP REQUEST (req 0xc00010e200)
2019/11/06 10:47:48 DEBUG : GET /drive/v3/files/root?alt=json&fields=id&prettyPrint=false&supportsAllDrives=true HTTP/1.1
Host: www.googleapis.com
User-Agent: animosityapp
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.11.0 gdcl/20190802
Accept-Encoding: gzip

2019/11/06 10:47:48 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2019/11/06 10:47:48 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2019/11/06 10:47:48 DEBUG : HTTP RESPONSE (req 0xc00010e200)
2019/11/06 10:47:48 DEBUG : HTTP/1.1 200 OK
Transfer-Encoding: chunked

Is the PI doing any DNS trickery or other services running to intercept the request out?

bullishpip · November 6, 2019, 4:15pm

Not to my knowledge (no Raspberry trickery - bone stock build).

The Raspberry Pi build is a fresh build without any modifications.

The process I am using is:

Build Raspberry Pi using Raspberry Pi build process: https://www.raspberrypi.org/documentation/installation/installing-images/README.md

hattps://www.raspberrypi.org/downloads/raspbian/

Install rclone using rclone install process:

Script download and install

To install rclone on Linux/macOS/BSD systems, run:

curl https://rclone.org/install.sh | sudo bash

Before roughly two days ago, using the above two steps resulted in rclone working without issue, on a Raspberry Pi Zero W.

Today, the same exact process results in the 404 error.

Animosity022 · November 6, 2019, 4:17pm

I don't see anything else obvious to me as it's connecting to Google and it seems to be returning the 404.

@ncw - do you see something else that I'm missing?

ncw · November 6, 2019, 9:59pm

I suspect there is something else in the config file.

Try deleting the remote (using rclone config) then remaking it to remove any old config.

bullishpip · November 7, 2019, 5:06pm

How would this be possible (something else in the rclone config file)?

The build process starts with a complete fresh disk (newly formatted), and a fresh rclone install.

If you look at the rclone config file it is literally only 4 lines.

bullishpip · November 7, 2019, 5:16pm

There is the possibility it is a Google Drive problem, or Raspberry Pi problem not an rclone problem.

Can someone else verify they can use rclone 1.50.1 on a Raspberry Pi Zero W today (11/07/2019)?

bullishpip · November 7, 2019, 11:34pm

I just testing on a second Raspberry Pi Zero W and am getting the same 404 error message.

root@pi0-tm3:~# rclone ls gdrive:

2019/11/07 22:58:20 Failed to ls: googleapi: Error 404: File not found: 0ABUrlFLwJwDzUk9PVA., notFound

This exact same Raspberry Pi was working just fine 5 days ago.

So either there is a problem with rclone, Google Drive or my google account.

ncw · November 8, 2019, 11:42am

I managed to re-create this issue.

It is something to do with the drive.file scope and rclone looking for the root folder ID...

I had a go at a fix here - can you try it?

https://beta.rclone.org/branch/v1.50.1-021-g223fd798-fix-drive-files-scope-beta/ (uploaded in 15-30 mins)

bullishpip · November 8, 2019, 3:26pm

This is great news. Thank you @ncw!

I am traveling the next couple of days and so won't be able to test this fix until like Monday.

As soon as I get back I will test. Thanks again for the support!