(Google to Google) Error 404: File not found

What is the problem you are having with rclone?

Attempting a server-side, Google-to-Google move results in an error:

googleapi: Error 404: File not found

What is your rclone version (output from rclone version)

rclone v1.53.3

  • os/arch: windows/amd64
  • go version: go1.15.5

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10, 64 bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive (source and target, different domains)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone move SourceBR: TargetBR: --drive-server-side-across-configs -vv

The rclone config contents with secrets removed.

[SourceBR]
type = drive
scope = drive
client_id = ***
service_account_file = C:\Users\Chad\.config\rclone\source.json

[TargetBR]
type = drive
scope = drive
client_id = ***
service_account_file = C:\Users\Chad\.config\rclone\target.json

A log from the command with the -vv flag

2021/01/22 01:39:35 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "move" "SourceBR:" "TargetBR:" "--drive-server-side-across-configs" "-vv"]
2021/01/22 01:39:35 DEBUG : Using config file from "C:\\Users\\Chad\\.config\\rclone\\rclone.conf"
2021/01/22 01:39:35 DEBUG : Creating backend with remote "SourceBR:"
2021/01/22 01:39:36 DEBUG : Google drive root '': root_folder_id = "0AI3Ru7siWxXlUk9PVA" - save this in the config to speed up startup
2021/01/22 01:39:36 DEBUG : Creating backend with remote "TargetBR:"
2021/01/22 01:39:36 DEBUG : Google drive root '': root_folder_id = "0AIpCWpkVAFFoUk9PVA" - save this in the config to speed up startup
2021/01/22 01:39:37 DEBUG : Google drive root '': Waiting for checks to finish
2021/01/22 01:39:37 DEBUG : Google drive root '': Waiting for transfers to finish
2021/01/22 01:39:37 ERROR : Copy of Round2 Doc.docx: Couldn't move: googleapi: Error 404: File not found: 0AI3Ru7siWxXlUk9PVA., notFound
2021/01/22 01:39:37 ERROR : Round2 Doc.docx: Couldn't move: googleapi: Error 404: File not found: 0AI3Ru7siWxXlUk9PVA., notFound
2021/01/22 01:39:37 ERROR : Copy2 of Top 5 Analysis.pptx: Couldn't move: googleapi: Error 404: File not found: 0AI3Ru7siWxXlUk9PVA., notFound
2021/01/22 01:39:37 INFO  : There was nothing to transfer
2021/01/22 01:39:37 ERROR : Attempt 1/3 failed with 3 errors and: googleapi: Error 404: File not found: 0AI3Ru7siWxXlUk9PVA., notFound
2021/01/22 01:39:37 ERROR : Copy of Round2 Doc.docx: Couldn't move: googleapi: Error 404: File not found: 0AI3Ru7siWxXlUk9PVA., notFound
2021/01/22 01:39:37 ERROR : Copy2 of Top 5 Analysis.pptx: Couldn't move: googleapi: Error 404: File not found: 0AI3Ru7siWxXlUk9PVA., notFound
2021/01/22 01:39:37 ERROR : Round2 Doc.docx: Couldn't move: googleapi: Error 404: File not found: 0AI3Ru7siWxXlUk9PVA., notFound
2021/01/22 01:39:37 DEBUG : Google drive root '': Waiting for checks to finish
2021/01/22 01:39:37 DEBUG : Google drive root '': Waiting for transfers to finish
2021/01/22 01:39:37 INFO  : There was nothing to transfer
2021/01/22 01:39:37 ERROR : Attempt 2/3 failed with 3 errors and: googleapi: Error 404: File not found: 0AI3Ru7siWxXlUk9PVA., notFound
2021/01/22 01:39:38 ERROR : Copy2 of Top 5 Analysis.pptx: Couldn't move: googleapi: Error 404: File not found: 0AI3Ru7siWxXlUk9PVA., notFound
2021/01/22 01:39:38 ERROR : Round2 Doc.docx: Couldn't move: googleapi: Error 404: File not found: 0AI3Ru7siWxXlUk9PVA., notFound
2021/01/22 01:39:38 ERROR : Copy of Round2 Doc.docx: Couldn't move: googleapi: Error 404: File not found: 0AI3Ru7siWxXlUk9PVA., notFound
2021/01/22 01:39:38 DEBUG : Google drive root '': Waiting for checks to finish
2021/01/22 01:39:38 DEBUG : Google drive root '': Waiting for transfers to finish
2021/01/22 01:39:38 INFO  : There was nothing to transfer
2021/01/22 01:39:38 ERROR : Attempt 3/3 failed with 3 errors and: googleapi: Error 404: File not found: 0AI3Ru7siWxXlUk9PVA., notFound
2021/01/22 01:39:38 INFO  :
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 3 (retrying may help)
Checks:                 9 / 9, 100%
Elapsed time:         3.3s

2021/01/22 01:39:38 DEBUG : 16 go routines active
2021/01/22 01:39:38 Failed to move with 3 errors: last error was: googleapi: Error 404: File not found: 0AI3Ru7siWxXlUk9PVA., notFound

I have the impersonation users set in the environment:

RCLONE_CONFIG_SOURCEBR_IMPERSONATE=user@sourcebr.com
RCLONE_CONFIG_TARGETBR_IMPERSONATE=user@targetbr.com

I understand that the 404 error typically means that the target user doesn't have access to the files I am attempting to move. That's not the case here, though.

λ rclone lsf SourceBR:
Copy of Round2 Doc.docx
Copy2 of Top 5 Analysis.pptx
Round2 Doc.docx
subdir/

λ rclone lsf TargetBR: --drive-shared-with-me
Copy2 of Top 5 Analysis.pptx
Round2 Doc.docx

Since user@targetbr.com can access the two files from SourceBR: that are shared with it, I expect those files to move and not throw 404 errors.

I'd double check you have permission setup like this:

as that's what a 404 means.

I have two separate scenarios involving a total of four remotes, two source and two target. When I opened this issue, I could reproduce the problem reported here in both scenarios, one of which is the data included in this topic.

In both scenarios I can run lsf with the --drive-shared-with-me option on the target and see the files in the source that I want moved. Does that not rule out a permissions issue? The answer you linked to refers to upgrading Content Manager access to Manager, but that is for Team Drives, which is not part of my scenario. I am dealing strictly with My Drives.

What I have found since opening this issue is that for one of the scenarios--two different remotes named Source: and Target:--the problem is resolved by adding --drive-root-folder-id "root". I don't know why, but it appears Rclone cannot access files at the root of My Drive without that option set. Any background information to explain that is appreciated.

For the other scenario, I now get a different error when passing in --drive-root-folder-id "root".

λ rclone move SourceBR: TargetBR: --drive-server-side-across-configs --drive-root-folder-id "root" -vv
2021/01/22 09:58:01 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "move" "SourceBR:" "TargetBR:" "--drive-server-side-across-configs" "--drive-root-folder-id" "root" "-vv"]
2021/01/22 09:58:01 DEBUG : Using config file from "C:\\Users\\Chad\\.config\\rclone\\rclone.conf"
2021/01/22 09:58:01 DEBUG : Creating backend with remote "SourceBR:"
2021/01/22 09:58:01 DEBUG : Creating backend with remote "TargetBR:"
2021/01/22 09:58:02 ERROR : Copy of Round2 Doc.docx: Couldn't move: googleapi: Error 403: The user does not have sufficient permissions for this file., insufficientFilePermissions
2021/01/22 09:58:02 DEBUG : Google drive root '': Waiting for checks to finish
2021/01/22 09:58:02 DEBUG : Google drive root '': Waiting for transfers to finish
2021/01/22 09:58:02 INFO  : Round2 Doc.docx: Moved (server side)
2021/01/22 09:58:03 INFO  : Copy2 of Top 5 Analysis.pptx: Moved (server side)
2021/01/22 09:58:03 INFO  : There was nothing to transfer
2021/01/22 09:58:03 ERROR : Attempt 1/3 failed with 1 errors and: googleapi: Error 403: The user does not have sufficient permissions for this file., insufficientFilePermissions
2021/01/22 09:58:03 DEBUG : Google drive root '': Waiting for checks to finish
2021/01/22 09:58:03 DEBUG : Google drive root '': Waiting for transfers to finish
2021/01/22 09:58:03 ERROR : Copy of Round2 Doc.docx: Couldn't move: googleapi: Error 403: The user does not have sufficient permissions for this file., insufficientFilePermissions
2021/01/22 09:58:03 INFO  : Round2 Doc.docx: Moved (server side)
2021/01/22 09:58:04 INFO  : Copy2 of Top 5 Analysis.pptx: Moved (server side)
2021/01/22 09:58:04 INFO  : There was nothing to transfer
2021/01/22 09:58:04 ERROR : Attempt 2/3 failed with 1 errors and: googleapi: Error 403: The user does not have sufficient permissions for this file., insufficientFilePermissions
2021/01/22 09:58:04 DEBUG : Google drive root '': Waiting for checks to finish
2021/01/22 09:58:04 DEBUG : Google drive root '': Waiting for transfers to finish
2021/01/22 09:58:04 ERROR : Copy of Round2 Doc.docx: Couldn't move: googleapi: Error 403: The user does not have sufficient permissions for this file., insufficientFilePermissions
2021/01/22 09:58:04 INFO  : Round2 Doc.docx: Moved (server side)
2021/01/22 09:58:04 INFO  : Copy2 of Top 5 Analysis.pptx: Moved (server side)
2021/01/22 09:58:04 INFO  : There was nothing to transfer
2021/01/22 09:58:04 ERROR : Attempt 3/3 failed with 1 errors and: googleapi: Error 403: The user does not have sufficient permissions for this file., insufficientFilePermissions
2021/01/22 09:58:04 INFO  :
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 1 (retrying may help)
Checks:                 9 / 9, 100%
Renamed:                6
Elapsed time:         3.4s

2021/01/22 09:58:04 DEBUG : 16 go routines active
2021/01/22 09:58:04 Failed to move: googleapi: Error 403: The user does not have sufficient permissions for this file., insufficientFilePermissions

λ rclone lsf TargetBR:
Copy of Top 5 Analysis.pptx
SheetOStuff.xlsx
subdir/

Note that even though there's an indication in the logging that the files were moved, they were not. I understand that HTTP 403 again points to permissions, but I am still left wondering why I can see the files in the source that I want moved. Any idea how I can determine exactly what permissions are needed but are lacking? The impersonation user in TargetBR: has Editor access to the files in SourceBR:, and that is the maximum level of access I can provide.

I need to amend what I said in my prior post. The output from Rclone made it appear that adding --drive-root-folder-id "root" worked for one of my scenarios, but all that seems to have occurred is that I no longer get errors. The file doesn't actually get moved.

λ rclone move Source: Target: --drive-server-side-across-configs --drive-root-folder-id "root" -vv
2021/01/22 10:05:04 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "move" "Source:" "Target:" "--drive-server-side-across-configs" "--drive-root-folder-id" "root" "-vv"]
2021/01/22 10:05:04 DEBUG : Using config file from "C:\\Users\\Chad\\.config\\rclone\\rclone.conf"
2021/01/22 10:05:04 DEBUG : Creating backend with remote "Source:"
2021/01/22 10:05:04 DEBUG : Creating backend with remote "Target:"
2021/01/22 10:05:05 DEBUG : Google drive root '': Waiting for checks to finish
2021/01/22 10:05:05 DEBUG : Google drive root '': Waiting for transfers to finish
2021/01/22 10:05:06 INFO  : Doc 1.docx: Moved (server side)
2021/01/22 10:05:06 INFO  : There was nothing to transfer
2021/01/22 10:05:06 INFO  :
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks:                 1 / 1, 100%
Renamed:                1
Elapsed time:         1.5s

2021/01/22 10:05:06 DEBUG : 12 go routines active

C:\Temp
λ rclone lsf Source:
Doc 1.docx
Migration/

C:\Temp
λ rclone lsf Target:
Do not share/
Migration/
Sync Test 1.docx
dummy.pdf

Maybe a silly question, but are you actually using "rootid" for the RootID or you just didn't feel like posting the actual ID?

I'm literally using "root" in this case, which is an alias to the root folder ID per the Google docs: https://developers.google.com/drive/api/v3/folder.

Yep, sorry as I typo'ed the rootid and meant root.

Was trying to make sure I got it right as I was trying to replicate what you had going on.

I ran these commands consecutively to show the difference.

λ rclone move Source: Target: --drive-server-side-across-configs -vv
2021/01/22 10:53:14 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "move" "Source:" "Target:" "--drive-server-side-across-configs" "-vv"]
2021/01/22 10:53:14 DEBUG : Using config file from "C:\\Users\\Chad\\.config\\rclone\\rclone.conf"
2021/01/22 10:53:14 DEBUG : Creating backend with remote "Source:"
2021/01/22 10:53:14 DEBUG : Google drive root '': root_folder_id = "0ANK1xd3sAx84Uk9PVA" - save this in the config to speed up startup
2021/01/22 10:53:14 DEBUG : Creating backend with remote "Target:"
2021/01/22 10:53:15 DEBUG : Google drive root '': root_folder_id = "0ALcRX0-WmpNVUk9PVA" - save this in the config to speed up startup
2021/01/22 10:53:15 DEBUG : Google drive root '': Waiting for checks to finish
2021/01/22 10:53:15 DEBUG : Google drive root '': Waiting for transfers to finish
2021/01/22 10:53:15 ERROR : Doc 1.docx: Couldn't move: googleapi: Error 404: File not found: 0ANK1xd3sAx84Uk9PVA., notFound
2021/01/22 10:53:15 INFO  : There was nothing to transfer
2021/01/22 10:53:15 ERROR : Attempt 1/3 failed with 1 errors and: googleapi: Error 404: File not found: 0ANK1xd3sAx84Uk9PVA., notFound
2021/01/22 10:53:15 DEBUG : Google drive root '': Waiting for checks to finish
2021/01/22 10:53:15 DEBUG : Google drive root '': Waiting for transfers to finish
2021/01/22 10:53:16 ERROR : Doc 1.docx: Couldn't move: googleapi: Error 404: File not found: 0ANK1xd3sAx84Uk9PVA., notFound
2021/01/22 10:53:16 INFO  : There was nothing to transfer
2021/01/22 10:53:16 ERROR : Attempt 2/3 failed with 1 errors and: googleapi: Error 404: File not found: 0ANK1xd3sAx84Uk9PVA., notFound
2021/01/22 10:53:16 DEBUG : Google drive root '': Waiting for checks to finish
2021/01/22 10:53:16 DEBUG : Google drive root '': Waiting for transfers to finish
2021/01/22 10:53:16 ERROR : Doc 1.docx: Couldn't move: googleapi: Error 404: File not found: 0ANK1xd3sAx84Uk9PVA., notFound
2021/01/22 10:53:16 INFO  : There was nothing to transfer
2021/01/22 10:53:16 ERROR : Attempt 3/3 failed with 1 errors and: googleapi: Error 404: File not found: 0ANK1xd3sAx84Uk9PVA., notFound
2021/01/22 10:53:16 INFO  :
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 1 (retrying may help)
Checks:                 3 / 3, 100%
Elapsed time:         2.1s

2021/01/22 10:53:16 DEBUG : 10 go routines active
2021/01/22 10:53:16 Failed to move: googleapi: Error 404: File not found: 0ANK1xd3sAx84Uk9PVA., notFound

C:\Temp
λ rclone move Source: Target: --drive-server-side-across-configs --drive-root-folder-id "root" -vv
2021/01/22 10:53:26 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "move" "Source:" "Target:" "--drive-server-side-across-configs" "--drive-root-folder-id" "root" "-vv"]
2021/01/22 10:53:26 DEBUG : Using config file from "C:\\Users\\Chad\\.config\\rclone\\rclone.conf"
2021/01/22 10:53:26 DEBUG : Creating backend with remote "Source:"
2021/01/22 10:53:26 DEBUG : Creating backend with remote "Target:"
2021/01/22 10:53:27 DEBUG : Google drive root '': Waiting for checks to finish
2021/01/22 10:53:27 DEBUG : Google drive root '': Waiting for transfers to finish
2021/01/22 10:53:28 INFO  : Doc 1.docx: Moved (server side)
2021/01/22 10:53:28 INFO  : There was nothing to transfer
2021/01/22 10:53:28 INFO  :
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks:                 1 / 1, 100%
Renamed:                1
Elapsed time:         1.1s

2021/01/22 10:53:28 DEBUG : 10 go routines active

I still have the problem where the file isn't being moved (no idea or clues in the log as to why), but at least I don't get the 404 errors.

So the first one, you are back at permission not setup right for the service accounts as it's using the right root IDs for each drive.

The second, I'm surmising it never leaves the first drive as the root IDs are getting confused by using 'root' in there, but I need to dig into the requests a bit and see.

If you want to test scenario one, turn off server side moves and if it works, you know you have a permissions issue.

I hear what you are saying, but what permissions issue could it be when the following are the case:

  • The target user can see the files in the source using --drive-shared-with-me.
  • The sharing level for the target user is the maximum possible (Editor).

When doing a move, who is doing what? Is it the source user attempting to write the file it owns to the target user's My Drive? Is it the target user attempting to read the file it has been given shared access to and write it to its own My Drive? I suspect it's the latter, and my tests have already confirmed that the user can read the file. Writing to your own My Drive is an implicit permission, so I'm confused as to what permissions this could possibly be.

It's detailed out in the link I had shared above on what is needed.

I'd test with the permission suggested there and see how it works.

As I mentioned earlier in the thread, that link is for Team Drives, which I am not using here. There is no concept of Manager and Content Manager for My Drives.

My apologies as you did state that as I sometimes mistake threads and get confused so that's my bad.

I can't validate that item on my side then as I only have 1 GSuite.

No worries at all. I appreciate the help! If you are able to DM me and willing to test, I can give you access to my GSuite.

The second, I'm surmising it never leaves the first drive as the root IDs are getting confused by using 'root' in there, but I need to dig into the requests a bit and see.

Yes, I believe that is what is happening, shown here:

2021/01/22 11:51:10 DEBUG : HTTP REQUEST (req 0xc0006a4200)
2021/01/22 11:51:10 DEBUG : PATCH /drive/v3/files/1yJ9k5nL0oGlD2Nb1sRYsVuHzg0i39RUT?addParents=root&alt=json&fields=id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks&prettyPrint=false&removeParents=root&supportsAllDrives=true HTTP/1.1

I'm not positive, but I suspect addParents=root is effectively a no-op in this situation.

Yep, I sent a PM in regards to the GSuite and tagged Nick as well as we can figure out what's the best from there. Ignore the fact I toggled the post as I was trying to link it and made it private for a few seconds there..

I think I replied to the private version, as I don't see my response here.

Here's what I posted in the private thread:

I think I see the problem. High-level recap of the scenario:

  1. Two My Drive users in different domains configured with remotes Source: and Target: and appropriate impersonation for each.
  2. Source: user shares a file in My Drive root with Target: user.
  3. Attempt a server-side move of Source: to Target:.

When you do this, you will see this PATCH request/response:

2021/01/22 12:11:51 DEBUG : Creating backend with remote "Source:"
2021/01/22 12:11:51 DEBUG : Google drive root '': **root_folder_id = "0ANK1xd3sAx84Uk9PVA"** - save this in the config to speed up startup

2021/01/22 12:11:52 DEBUG : HTTP REQUEST (req 0xc000610800)
2021/01/22 12:11:52 DEBUG : PATCH /drive/v3/files/1yJ9k5nL0oGlD2Nb1sRYsVuHzg0i39RUT?addParents=0ALcRX0-WmpNVUk9PVA&alt=json&fields=id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks&prettyPrint=false&**removeParents=0ANK1xd3sAx84Uk9PVA**&supportsAllDrives=true HTTP/1.1
Host: www.googleapis.com
User-Agent: rclone/v1.53.3
Content-Length: 97
Authorization: XXXX
Content-Type: application/json
X-Goog-Api-Client: gl-go/1.15.5 gdcl/20200616
Accept-Encoding: gzip

{"description":"sample.txt","modifiedTime":"2021-01-22T17:16:55.392000000Z","name":"sample.txt"}
2021/01/22 12:11:52 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/01/22 12:11:52 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/01/22 12:11:52 DEBUG : HTTP RESPONSE (req 0xc000610800)
2021/01/22 12:11:52 DEBUG : HTTP/1.1 404 Not Found
Transfer-Encoding: chunked
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private, max-age=0
Content-Security-Policy: frame-ancestors 'self'
Content-Type: application/json; charset=UTF-8
Date: Fri, 22 Jan 2021 18:11:51 GMT
Expires: Fri, 22 Jan 2021 18:11:51 GMT
Server: GSE
Vary: Origin
Vary: X-Origin
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block

da
{"error":{"errors":[{"domain":"global","reason":"notFound","message":"File not found: 0ANK1xd3sAx84Uk9PVA.","locationType":"parameter","location":"fileId"}],"code":404,"message":**"File not found: 0ANK1xd3sAx84Uk9PVA."**}}
0

So the issue is that the Target: user has no access to the root of the Source: My Drive and thus has no permission to remove that root folder ID as a parent, completing the move. Given that I don't think you can grant permission to the root of My Drive, I'm not sure I see how this is supposed to work.

Can this operation be broken into two steps?

  1. Remote: user issues PATCH to add its parent.
  2. Source: user issues PATCH to remove its parent.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.