Copying files from a publicly accessible google drive folder added as a shortcut getting error 404, server side copies are disabled

Suspected Bug
1

What is the problem you are having with rclone?

Getting error 404 when copying files from one google drive remote to another, the folder that I am trying to copy is a publically accessible google drive folder, this folder hasn't been shared directly with me, and as a result does not appear in "Shared with me" in the Google Drive Website UI, as a result I have added the publicly accessible google drive folder as a shortcut within my own Google Drive.

Permissions Bug

I did some searching here and found it's a permissions thing, so below is two log outputs, the 2nd has --disable copy but I still get error 404. Is this a bug? Does this mean I am out of luck, and would need to do it manually?

I can download the file showing in the error using the Google Drive Website.

Run the command 'rclone version' and share the full output of the command.

rclone v1.58.1
- os/version: debian 9.13 (64 bit)
- os/kernel: 4.19.123-feral (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.17.9
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Command 1 - Server Side Copies Enabled

./bin/rclone copy GD:'/WADs/Old/cIOS' GD:'/Emulation/WADs/Old/cIOS' --backup-dir GD:'/Duplicates/Emulation/WADs/Old/cIOS' --buffer-size 500M --checkers 2 --check-first --checksum --drive-chunk-size 256M --drive-export-formats 'docx,xlsx,pptx,svg,pdf' --drive-pacer-min-sleep 100ms --fast-list --log-level DEBUG --log-file './temp.log' --low-level-retries 1 --order-by 'modtime,asc' --progress --retries 1 --retries-sleep 2s --stats 0 --stats-file-name-length 70 --tpslimit 2 --tpslimit-burst 2 --transfers 2 --use-mmap --user-agent 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36'

Command 2 - Server Side Copies Disabled

rclone copy GD:'/WADs/Old/cIOS' GD:'/Emulation/WADs/Old/cIOS' --buffer-size 500M --checkers 2 --check-first --checksum --disable move,copy --drive-chunk-size 256M --drive-export-formats 'docx,xlsx,pptx,svg,pdf' --drive-pacer-min-sleep 100ms --drive-server-side-across-configs --fast-list --log-level DEBUG --log-file './temp.log' --low-level-retries 1 --order-by 'modtime,asc' --progress --retries 1 --retries-sleep 2s --stats 0 --stats-file-name-length 70 --tpslimit 2 --tpslimit-burst 2 --transfers 2 --use-mmap --user-agent 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36'

The rclone config contents with secrets removed.

[GD]
type = drive
client_id = **REDACTED**.apps.googleusercontent.com
client_secret = **REDACTED**
token = {"access_token":"**REDACTED**","token_type":"Bearer","refresh_token":"**REDACTED**","expiry":"2022-05-07T18:14:20.002423138Z"}
root_folder_id = **REDACTED**

This is a listing of the folder I am wanting to copy, it only has files in it and contains no sub folders.

2022/05/07 17:54:27 DEBUG : Using config file from "/path/to/.config/rclone/rclone.conf"
2022/05/07 17:54:27 DEBUG : rclone: Version "v1.55.0" starting with parameters ["./bin/rclone" "ls" "GD:/WADs/Old/cIOS" "--fast-list" "--log-level" "DEBUG"]
2022/05/07 17:54:27 DEBUG : Creating backend with remote "GD:/WADs/Old/cIOS"
2022/05/07 17:54:29 DEBUG : fs cache: renaming cache item "GD:/WADs/Old/cIOS" to be canonical "GD:WADs/Old/cIOS"
    24496 BC-NUS-v6.wad
2022/05/07 17:54:29 DEBUG : 4 go routines active

A log from the command with the -vv flag

Command 1 Output - Server Side Copies Enabled

2022/05/07 17:18:12 DEBUG : Using config file from "/path/to/.config/rclone/rclone.conf"
2022/05/07 17:18:12 INFO  : Starting transaction limiter: max 2 transactions/s with burst 2
2022/05/07 17:18:12 DEBUG : rclone: Version "v1.55.0" starting with parameters ["./bin/rclone" "copy" "GD:/WADs/Old/cIOS" "GD:/Emulation/WADs/Old/cIOS" "--backup-dir" "GD:/Duplicates/Emulation/WADs/Old/cIOS" "--buffer-size" "500M" "--checkers" "2" "--check-first" "--checksum" "--drive-chunk-size" "256M" "--drive-export-formats" "docx,xlsx,pptx,svg,pdf" "--drive-pacer-min-sleep" "100ms" "--drive-server-side-across-configs" "--fast-list" "--log-level" "DEBUG" "--log-file" "./temp.log" "--low-level-retries" "1" "--order-by" "modtime,asc" "--progress" "--retries" "1" "--retries-sleep" "2s" "--stats" "0" "--stats-file-name-length" "70" "--tpslimit" "2" "--tpslimit-burst" "2" "--transfers" "2" "--use-mmap" "--user-agent" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36"]
2022/05/07 17:18:12 DEBUG : Creating backend with remote "GD:/WADs/Old/cIOS"
2022/05/07 17:18:12 DEBUG : GD: detected overridden config - adding "{6wVkM}" suffix to name
2022/05/07 17:18:14 DEBUG : fs cache: renaming cache item "GD:/WADs/Old/cIOS" to be canonical "GD{6wVkM}:WADs/Old/cIOS"
2022/05/07 17:18:14 DEBUG : Creating backend with remote "GD:/Emulation/WADs/Old/cIOS"
2022/05/07 17:18:14 DEBUG : GD: detected overridden config - adding "{6wVkM}" suffix to name
2022/05/07 17:18:16 DEBUG : fs cache: renaming cache item "GD:/Emulation/WADs/Old/cIOS" to be canonical "GD{6wVkM}:Emulation/WADs/Old/cIOS"
2022/05/07 17:18:16 INFO  : Google drive root 'Emulation/WADs/Old/cIOS': Running all checks before starting transfers
2022/05/07 17:18:16 DEBUG : Creating backend with remote "GD:/Duplicates/Emulation/WADs/Old/cIOS"
2022/05/07 17:18:16 DEBUG : GD: detected overridden config - adding "{6wVkM}" suffix to name
2022/05/07 17:18:18 DEBUG : fs cache: renaming cache item "GD:/Duplicates/Emulation/WADs/Old/cIOS" to be canonical "GD{6wVkM}:Duplicates/Emulation/WADs/Old/cIOS"
2022/05/07 17:18:19 DEBUG : Google drive root 'Emulation/WADs/Old/cIOS': Waiting for checks to finish
2022/05/07 17:18:19 INFO  : Google drive root 'Emulation/WADs/Old/cIOS': Checks finished, now starting transfers
2022/05/07 17:18:19 DEBUG : Google drive root 'Emulation/WADs/Old/cIOS': Waiting for transfers to finish
2022/05/07 17:18:21 ERROR : BC-NUS-v6.wad: Failed to copy: googleapi: Error 404: File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k., notFound
2022/05/07 17:18:21 ERROR : Attempt 1/1 failed with 1 errors and: googleapi: Error 404: File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k., notFound
2022/05/07 17:18:23 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 1 (retrying may help)
Checks:                20 / 20, 100%
Elapsed time:        10.7s

2022/05/07 17:18:23 DEBUG : 8 go routines active
2022/05/07 17:18:23 Failed to copy: googleapi: Error 404: File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k., notFound

Command 2 Output - Server Side Copies Disabled

2022/05/07 17:36:34 DEBUG : Using config file from "/path/to/.config/rclone/rclone.conf"
2022/05/07 17:36:34 INFO  : Starting transaction limiter: max 2 transactions/s with burst 2
2022/05/07 17:36:34 DEBUG : rclone: Version "v1.55.0" starting with parameters ["./bin/rclone" "copy" "GD:/WADs/Old/cIOS" "GD:/Emulation/WADs/Old/cIOS" "--buffer-size" "500M" "--checkers" "2" "--check-first" "--checksum" "--disable" "move,copy" "--drive-chunk-size" "256M" "--drive-export-formats" "docx,xlsx,pptx,svg,pdf" "--drive-pacer-min-sleep" "100ms" "--drive-server-side-across-configs" "--fast-list" "--log-level" "DEBUG" "--log-file" "./temp.log" "--low-level-retries" "1" "--order-by" "modtime,asc" "--progress" "--retries" "1" "--retries-sleep" "2s" "--stats" "0" "--stats-file-name-length" "70" "--tpslimit" "2" "--tpslimit-burst" "2" "--transfers" "2" "--use-mmap" "--user-agent" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36"]
2022/05/07 17:36:34 DEBUG : Creating backend with remote "GD:/WADs/Old/cIOS"
2022/05/07 17:36:34 DEBUG : GD: detected overridden config - adding "{6wVkM}" suffix to name
2022/05/07 17:36:34 DEBUG : Reset feature "move"
2022/05/07 17:36:34 DEBUG : Reset feature "copy"
2022/05/07 17:36:37 DEBUG : fs cache: renaming cache item "GD:/WADs/Old/cIOS" to be canonical "GD{6wVkM}:WADs/Old/cIOS"
2022/05/07 17:36:37 DEBUG : Creating backend with remote "GD:/Emulation/WADs/Old/cIOS"
2022/05/07 17:36:37 DEBUG : GD: detected overridden config - adding "{6wVkM}" suffix to name
2022/05/07 17:36:37 DEBUG : Reset feature "move"
2022/05/07 17:36:37 DEBUG : Reset feature "copy"
2022/05/07 17:36:40 DEBUG : fs cache: renaming cache item "GD:/Emulation/WADs/Old/cIOS" to be canonical "GD{6wVkM}:Emulation/WADs/Old/cIOS"
2022/05/07 17:36:40 INFO  : Google drive root 'Emulation/WADs/Old/cIOS': Running all checks before starting transfers
2022/05/07 17:36:40 DEBUG : Google drive root 'Emulation/WADs/Old/cIOS': Waiting for checks to finish
2022/05/07 17:36:40 INFO  : Google drive root 'Emulation/WADs/Old/cIOS': Checks finished, now starting transfers
2022/05/07 17:36:40 DEBUG : Google drive root 'Emulation/WADs/Old/cIOS': Waiting for transfers to finish
2022/05/07 17:36:41 ERROR : BC-NUS-v6.wad: Failed to copy: failed to open source object: open file failed: googleapi: Error 404: File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k., notFound
2022/05/07 17:36:41 ERROR : Attempt 1/1 failed with 1 errors and: failed to open source object: open file failed: googleapi: Error 404: File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k., notFound
2022/05/07 17:36:43 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 1 (retrying may help)
Checks:                20 / 20, 100%
Elapsed time:         8.4s

2022/05/07 17:36:43 DEBUG : 6 go routines active
2022/05/07 17:36:43 Failed to copy: failed to open source object: open file failed: googleapi: Error 404: File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k., notFound
2

This (despite the 404 not found message) is usually a permissions error.

Just to make sure I understand the GD:'/WADs/Old/cIOS' source is a shortcut to a publically available folder - is that correct?

Presumably it lists properly in the web interface.

Can you try without --fast-list?

Also can you download the files locally with rclone?

3

@ncw

Yes I found some posts here mentioning that.

That is correct yes, it's not a folder that has been shared with me, there is a few files in that folder, some copy fine with server side copies, a few other files show error 404.

It does list properly in the web interface and I can download the files locally, that rclone is unable to copy to a different folder.

I tried that but the same thing happens when using --fast-list.

I hadn't thought to try that! But I have since tried and I still get error 404 when trying to download to local storage.

4

This sounds very similar to the other thread here:

That has some instructions for getting the full info about the file - can you try that? The ID you need is the one in the 404 message without the trailing ., so 0B7vPmuZfN3snV19LTEhUNXYzM0k.

The fact that the files don't download locally means that this isn't some server side copy oddity (of which there are plenty!).

5

@ncw I get an account restricted error when trying to get information about the file using the same account credentials that I am using with rclone.

Is the below correct or of any use to you?

2022/05/13 11:44:44 DEBUG : rclone: Version "v1.58.1" starting with parameters ["./bin/rclone" "copy" "GD:/WADs/Old/cIOS/BC-NUS-v6.wad" "." "-vv" "--dump" "bodies" "--low-level-retries" "1" "--retries" "1" "--log-file" "./temp.log"]
2022/05/13 11:44:44 DEBUG : Creating backend with remote "GD:/WADs/Old/cIOS/BC-NUS-v6.wad"
2022/05/13 11:44:44 DEBUG : Using config file from "/home/username/.config/rclone/rclone.conf"
2022/05/13 11:44:44 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2022/05/13 11:44:44 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/05/13 11:44:44 DEBUG : HTTP REQUEST (req 0xc000b00d00)
2022/05/13 11:44:44 DEBUG : GET /drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%270ADQjQMv9x0aDUk9PVA%27+in+parents%29+and+%28name%3D%27WADs%27%29+and+%28mimeType%3D%27application%2Fvnd.google-apps.folder%27+or+mimeType%3D%27application%2Fvnd.google-apps.shortcut%27%29&supportsAllDrives=true HTTP/1.1
Host: www.googleapis.com
User-Agent: rclone/v1.58.1
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.17.9 gdcl/20211027
Accept-Encoding: gzip

2022/05/13 11:44:51 DEBUG : HTTP RESPONSE (req 0xc000c12e00)
2022/05/13 11:44:51 DEBUG : HTTP/1.1 404 Not Found
Content-Length: 296
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private, max-age=0
Content-Type: application/json; charset=UTF-8
Date: Fri, 13 May 2022 11:44:51 GMT
Expires: Fri, 13 May 2022 11:44:51 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ADPycdt91fXrjqhT87IOEjBDHTT3RI7eFp5mX3GFgYcaXSZS4PTUa_uTFmLlBEtlp4HYEB_e6HxoFBidBTscch1-m5RU4A

{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "notFound",
    "message": "File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k.",
    "locationType": "parameter",
    "location": "fileId"
   }
  ],
  "code": 404,
  "message": "File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k."
 }
}

Using same account credentials as rclone.

Error 400: admin_policy_enforced
Account restricted

Request Details
The content in this section has been provided by the app developer. This content has not been reviewed or verified by Google.
If you’re the app developer, make sure that these request details comply with Google policies.
openid.realm:
response_type: permission id_token
redirect_uri: storagerelay://https/explorer.apis.google.com?id=auth450498
ss_domain: https://explorer.apis.google.com
client_id: 292824132082.apps.googleusercontent.com
gsiwebsdk: shim
access_type: online
scope: https://www.googleapis.com/auth/drive https://www.googleapis.com/auth/drive.appdata https://www.googleapis.com/auth/drive.file https://www.googleapis.com/auth/drive.metadata https://www.googleapis.com/auth/drive.metadata.readonly https://www.googleapis.com/auth/drive.photos.readonly https://www.googleapis.com/auth/drive.readonly openid

Using different account credentials.

{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "notFound",
    "message": "File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k.",
    "locationType": "parameter",
    "location": "fileId"
   }
  ],
  "code": 404,
  "message": "File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k."
 }
}
6

They all seem to be saying file not found in different ways!

Does it ever work? On the other thread the user reported that sometimes they were able to download the file and sometimes it gave 404, so the problem is somewhat intermittent.

7

@ncw

Sadly no it never ever works, I get error 404 100% of the time.

8

I'm guessing the source files must be restricted in some way I don't understand.

Do you get any different results, if you put the ID of the public folder as the root, so first this to list, then copy as before

rclone --drive-root-folder-id XXXXXX lsf GD:

I guess it is possible that going through the shortcut is causing a problem and trying it direct will test that.

9

@ncw

That worked, it listed all the files including those that I get error 404 on and can't copy, download.

Unfortunately it seems rclone copy doesn't work with specifying a root folder id instead of a remote and path.

10

You can specify it like this GD,root_folder_id=XXX: then you can use it in a server side copy

rclone copy GD,root_folder_id=XXX: GD:otherdir
11

@ncw

Oh I hadn't seen this message, I thought I had worked it out for myself.

This command sort of worked in that it let me run it.

2022/05/13 22:24:13 INFO  : Starting transaction limiter: max 2 transactions/s with burst 2
2022/05/13 22:24:13 DEBUG : rclone: Version "v1.58.1" starting with parameters ["./bin/rclone" "--drive-root-folder-id" "0B7vPmuZfN3sndy1PcXdMTVRUNGs" "copy" "GD:/" "/home/username/WADs/Old/cIOS" "--buffer-size" "500M" "--checkers" "2" "--check-first" "--checksum" "--drive-chunk-size" "256M" "--drive-export-formats" "docx,xlsx,pptx,svg,pdf" "--drive-pacer-min-sleep" "100ms" "--fast-list" "--log-level" "DEBUG" "--low-level-retries" "1" "--order-by" "modtime,asc" "--progress" "--retries" "1" "--retries-sleep" "2s" "--stats" "0" "--stats-file-name-length" "70" "--tpslimit" "2" "--tpslimit-burst" "2" "--transfers" "2" "--use-mmap" "--user-agent" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36" "--log-file" "./temp.log"]
2022/05/13 22:24:13 DEBUG : Creating backend with remote "GD:/"
2022/05/13 22:24:13 DEBUG : Using config file from "/home/username/.config/rclone/rclone.conf"
2022/05/13 22:24:13 DEBUG : GD: detected overridden config - adding "{0RHP3}" suffix to name
2022/05/13 22:24:13 DEBUG : fs cache: renaming cache item "GD:/" to be canonical "GD{0RHP3}:"
2022/05/13 22:24:13 DEBUG : Creating backend with remote "/home/username/WADs/Old/cIOS"
2022/05/13 22:24:13 INFO  : Local file system at /home/username/WADs/Old/cIOS: Running all checks before starting transfers
2022/05/13 22:24:14 DEBUG : Local file system at /home/username/WADs/Old/cIOS: Waiting for checks to finish
2022/05/13 22:24:14 INFO  : Local file system at /home/username/WADs/Old/cIOS: Checks finished, now starting transfers
2022/05/13 22:24:14 DEBUG : Local file system at /home/username/WADs/Old/cIOS: Waiting for transfers to finish
2022/05/13 22:24:14 ERROR : BC-NUS-v6.wad: Failed to copy: failed to open source object: open file failed: googleapi: Error 404: File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k., notFound
2022/05/13 22:24:14 ERROR : Attempt 1/1 failed with 1 errors and: failed to open source object: open file failed: googleapi: Error 404: File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k., notFound
2022/05/13 22:24:16 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Checks:                20 / 20, 100%
Elapsed time:         3.0s

2022/05/13 22:24:16 DEBUG : 5 go routines active
2022/05/13 22:24:16 Failed to copy: failed to open source object: open file failed: googleapi: Error 404: File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k., notFound

This is the command you posted above.

2022/05/13 22:38:47 INFO  : Starting transaction limiter: max 2 transactions/s with burst 2
2022/05/13 22:38:47 DEBUG : rclone: Version "v1.58.1" starting with parameters ["./bin/rclone" "copy" "GD,root_folder_id=0B7vPmuZfN3sndy1PcXdMTVRUNGs:" "GD:/Emulation/WADs/Old/cIOS" "--backup-dir" "GD:/Duplicates/Emulation/WADs/Old/cIOS" "--buffer-size" "500M" "--checkers" "2" "--check-first" "--checksum" "--drive-chunk-size" "256M" "--drive-export-formats" "docx,xlsx,pptx,svg,pdf" "--drive-pacer-min-sleep" "100ms" "--log-level" "DEBUG" "--log-file" "./temp.log" "--low-level-retries" "1" "--order-by" "modtime,asc" "--progress" "--retries" "1" "--retries-sleep" "2s" "--stats" "0" "--stats-file-name-length" "70" "--tpslimit" "2" "--tpslimit-burst" "2" "--transfers" "2" "--use-mmap" "--user-agent" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36"]
2022/05/13 22:38:47 DEBUG : Creating backend with remote "GD,root_folder_id=0B7vPmuZfN3sndy1PcXdMTVRUNGs:"
2022/05/13 22:38:47 DEBUG : Using config file from "/home/username/.config/rclone/rclone.conf"
2022/05/13 22:38:47 DEBUG : GD: detected overridden config - adding "{0RHP3}" suffix to name
2022/05/13 22:38:47 DEBUG : fs cache: renaming cache item "GD,root_folder_id=0B7vPmuZfN3sndy1PcXdMTVRUNGs:" to be canonical "GD{0RHP3}:"
2022/05/13 22:38:47 DEBUG : Creating backend with remote "GD:/Emulation/WADs/Old/cIOS"
2022/05/13 22:38:47 DEBUG : GD: detected overridden config - adding "{bR6Se}" suffix to name
2022/05/13 22:38:52 DEBUG : fs cache: renaming cache item "GD:/Emulation/WADs/Old/cIOS" to be canonical "GD{bR6Se}:Emulation/WADs/Old/cIOS"
2022/05/13 22:38:52 INFO  : Google drive root 'Emulation/WADs/Old/cIOS': Running all checks before starting transfers
2022/05/13 22:38:52 DEBUG : Creating backend with remote "GD:/Duplicates/Emulation/WADs/Old/cIOS"
2022/05/13 22:38:52 DEBUG : GD: detected overridden config - adding "{bR6Se}" suffix to name
2022/05/13 22:38:55 DEBUG : fs cache: renaming cache item "GD:/Duplicates/Emulation/WADs/Old/cIOS" to be canonical "GD{bR6Se}:Duplicates/Emulation/WADs/Old/cIOS"
2022/05/13 22:38:56 DEBUG : Google drive root 'Emulation/WADs/Old/cIOS': Waiting for checks to finish
2022/05/13 22:38:56 INFO  : Google drive root 'Emulation/WADs/Old/cIOS': Checks finished, now starting transfers
2022/05/13 22:38:56 DEBUG : Google drive root 'Emulation/WADs/Old/cIOS': Waiting for transfers to finish
2022/05/13 22:38:57 ERROR : BC-NUS-v6.wad: Failed to copy: failed to open source object: open file failed: googleapi: Error 404: File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k., notFound
2022/05/13 22:38:57 ERROR : Attempt 1/1 failed with 1 errors and: failed to open source object: open file failed: googleapi: Error 404: File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k., notFound
2022/05/13 22:38:59 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Checks:                20 / 20, 100%
Elapsed time:        11.7s

2022/05/13 22:38:59 DEBUG : 9 go routines active
2022/05/13 22:38:59 Failed to copy: failed to open source object: open file failed: googleapi: Error 404: File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k., notFound

Same command as above, except to local storage

2022/05/13 22:44:25 INFO  : Starting transaction limiter: max 2 transactions/s with burst 2
2022/05/13 22:44:25 DEBUG : rclone: Version "v1.58.1" starting with parameters ["./bin/rclone" "copy" "GD,root_folder_id=0B7vPmuZfN3sndy1PcXdMTVRUNGs:" "/home/username/Emulation/WADs/Old/cIOS" "--buffer-size" "500M" "--checkers" "2" "--check-first" "--checksum" "--drive-chunk-size" "256M" "--drive-export-formats" "docx,xlsx,pptx,svg,pdf" "--drive-pacer-min-sleep" "100ms" "--fast-list" "--log-level" "DEBUG" "--low-level-retries" "1" "--order-by" "modtime,asc" "--progress" "--retries" "1" "--retries-sleep" "2s" "--stats" "0" "--stats-file-name-length" "70" "--tpslimit" "2" "--tpslimit-burst" "2" "--transfers" "2" "--use-mmap" "--user-agent" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36" "--log-file" "./temp.log"]
2022/05/13 22:44:25 DEBUG : Creating backend with remote "GD,root_folder_id=0B7vPmuZfN3sndy1PcXdMTVRUNGs:"
2022/05/13 22:44:25 DEBUG : Using config file from "/home/username/.config/rclone/rclone.conf"
2022/05/13 22:44:25 DEBUG : GD: detected overridden config - adding "{0RHP3}" suffix to name
2022/05/13 22:44:25 DEBUG : fs cache: renaming cache item "GD,root_folder_id=0B7vPmuZfN3sndy1PcXdMTVRUNGs:" to be canonical "GD{0RHP3}:"
2022/05/13 22:44:25 DEBUG : Creating backend with remote "/home/username/Emulation/WADs/Old/cIOS"
2022/05/13 22:44:25 INFO  : Local file system at /home/username/Emulation/WADs/Old/cIOS: Running all checks before starting transfers
2022/05/13 22:44:26 DEBUG : Local file system at /home/username/Emulation/WADs/Old/cIOS: Waiting for checks to finish
2022/05/13 22:44:26 INFO  : Local file system at /home/username/Emulation/WADs/Old/cIOS: Checks finished, now starting transfers
2022/05/13 22:44:26 DEBUG : Local file system at /home/username/Emulation/WADs/Old/cIOS: Waiting for transfers to finish
2022/05/13 22:44:26 ERROR : BC-NUS-v6.wad: Failed to copy: failed to open source object: open file failed: googleapi: Error 404: File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k., notFound
2022/05/13 22:44:26 ERROR : Attempt 1/1 failed with 1 errors and: failed to open source object: open file failed: googleapi: Error 404: File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k., notFound
2022/05/13 22:44:28 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Checks:                20 / 20, 100%
Elapsed time:         3.1s

2022/05/13 22:44:28 DEBUG : 5 go routines active
2022/05/13 22:44:28 Failed to copy: failed to open source object: open file failed: googleapi: Error 404: File not found: 0B7vPmuZfN3snV19LTEhUNXYzM0k., notFound
12

All of those are giving the same error, essentially...

Is that the actual folder id? I tried using rclone lsf but I couldn't see any files in it.

13

This seems to be working in tests.

v1.59.0-beta.6141.d206761d2.fix-drive-resourcekey on branch fix-drive-resourcekey

This adds the resource key on download and server side copy which we weren't doing before.

1 Like
14

@ncw

Thanks for all your hard work and implementing this, as you mentioned it worked in my tests with no errors.

Thanks.

1 Like
15

I've merged this to master now which means it will be in the latest beta in 15-30 minutes and released in v1.59

1 Like
16

Awesome, thank you very much for fixing this so swiftly.

1 Like
17

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.