Gdrive to Gdrive Error 404 File not Found

t2m · April 17, 2019, 10:33pm

Since update rclone to the latest version 1.47 copy or sync between gdrives always result with googleapi error 404 files not found. Only Folders are created to the destination drive but not the files inside those folders. I Test this under both windows 10 and ubuntu. Fall back to the previous version 1.46 with the same command rclone copy g1: g2: work without error above.

rclone v1.47.0

os/arch: windows/amd64
go version: go1.11

DEBUG : pacer: Rate limited, increasing sleep to 16.907418935s
2019/04/17 15:22:14 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2019/04/17 15:22:14 DEBUG : pacer: Rate limited, increasing sleep to 16.111813216s
2019/04/17 15:22:15 ERROR : 3D Kanojo - Real Girl/3D Kanojo - Real Girl - 01 - About the Time I First Met Her.mkv: Failed to copy: googleapi: Error 404: File not found: 1uiCDbg9tMyxzdtiLdmfixUquf2XiS42h., notFound
2019/04/17 15:22:15 ERROR : 3D Kanojo - Real Girl/3D Kanojo - Real Girl - 03 - About the Time Normies Made Me Even Worse.mkv: Failed to copy: googleapi: Error 404: File not found: 120CISyLvslgr5Tnl2YxHEvbhOr1pg7a7., notFound
2019/04/17 15:22:15 ERROR : 3D Kanojo - Real Girl/3D Kanojo - Real Girl - 02 - About the Time My Chastity Was in a Predicament.mkv: Failed to copy: googleapi: Error 404: File not found: 13bwYGTW7iOKw8DVz9MZnQJCIJXGAQdH9., notFound
2019/04/17 15:22:15 ERROR : 3D Kanojo - Real Girl/3D Kanojo - Real Girl - 03 - About the Time Normies Made Me Even Worse.mp4: Failed to copy: googleapi: Error 404: File not found: 15wxWSnPVOJPBUW892-R6avo6IZzVB4Hy., notFound
2019/04/17 15:22:15 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=298954528890, userRateLimitExceeded)
2019/04/17 15:22:15 DEBUG : pacer: Rate limited, increasing sleep to 1.864856103s
2019/04/17 15:22:15 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=298954528890, userRateLimitExceeded)
2019/04/17 15:22:15 DEBUG : pacer: Rate limited, increasing sleep to 2.920737274s
2019/04/17 15:22:15 DEBUG : pacer: Reducing sleep to 0s
2019/04/17 15:22:15 ERROR : 3D Kanojo - Real Girl/3D Kanojo - Real Girl - 04 - About My Dark Times.mkv: Failed to copy: googleapi: Error 404: File not found: 1jbJ3uGB4lWbltnywhJEznXMXdBfsjP_5., notFound
2019/04/17 15:22:16 DEBUG : pacer: Reducing sleep to 0s
2019/04/17 15:22:18 ERROR : 3D Kanojo - Real Girl/3D Kanojo - Real Girl - 05 - About When I Tried to Make Summer Memories.mkv: Failed to copy: googleapi: Error 404: File not found: 1O6obXTjmy4d6hO__7U3VHRHL27dyLGja., notFound
2019/04/17 15:22:18 ERROR : 3D Kanojo - Real Girl/3D Kanojo - Real Girl - 06 - About the Way I Apologize.mkv: Failed to copy: googleapi: Error 404: File not found: 1Y7qYXnSETsVdx5PY3gW6sYBbIoAaR3pm., notFound
2019/04/17 15:22:18 ERROR : 3D Kanojo - Real Girl/3D Kanojo - Real Girl - 07 - About the Two Girls and Me.mkv: Failed to copy: googleapi: Error 404: File not found: 1vhKP7a1dDxEXQcOW_JfV1G0HD7TjXqWg., notFound
2019/04/17 15:22:18 ERROR : 3D Kanojo - Real Girl/3D Kanojo - Real Girl - 08 - About How Camping Was a High-Level Event for Me.mkv: Failed to copy: googleapi: Error 404: File not found: 1rmQLoqBmwD_Eu7DISiLV7KwWlRdzNCPj., notFound
2019/04/17 15:22:18 ERROR : 3D Kanojo - Real Girl/3D Kanojo - Real Girl - 09 - About Our Mutual Misunderstandings.mkv: Failed to copy: googleapi: Error 404: File not found: 1L8nYkx3rf22oR_BIw6zg1xKnuJpcG_F5., notFound

calisro · April 18, 2019, 12:15am

What does your config look like? (Secrets removed) and you really should get your own client ID to help with the rate limiting.

https://rclone.org/drive/#making-your-own-client-id

t2m · April 18, 2019, 2:12am

Rclone setting:

[g1]
type = drive
client_id =
client_secret =
token =

[g2]
type = drive
client_id =
client_secret =
token =

With the setting above I have no problem with all previous rclone versions. Until upgrade to 1.47 only remote folders are copy but not files within those folders. I even test out the lastest beta version 1.47 and it’s yield the same errors. Switch back to 1.46 copy and sync between remotes work fine. Maybe the new function server side copy causing these issue?

calisro · April 18, 2019, 2:29am

I’m not sure if it will help. But can you run that with

--dump-bodies

Also this is in the docs…

“Server side copies are also subject to a separate rate limit. If you see User rate limit exceeded errors, wait at least 24 hours and retry. You can disable server side copies with --disable copy to download and upload the files if you prefer.”

You could try disabling it…

igor_andreev · April 18, 2019, 7:43am

Thanks, it works for me. I have the same topicstarter’s problem and don’t read docs about server side copy.

t2m · April 18, 2019, 9:30am

–disable copy fix error 404 file not found . However, this defeat the beauty of server side copy. As both remotes g1 & g2 haven’t been sync or copy to for a few weeks so rate limit doesn’t apply here. I will wait another 24 hours to retest and report back the result. Thanks for the help

ncw · April 18, 2019, 9:42am

I see that --disable Copy fixes it which means that this is almost certainly a consequence of : https://github.com/ncw/rclone/commit/55b9a4ed309f6f442cde0a70d50dddc476e91243

Are the files owned by different users - is that what is causing the problem?

1.46 won’t have been doing server side copy between different remotes.

calisro · April 18, 2019, 12:07pm

I just checked my logs and I’m seeing this as well. All my files are owned by one account.

ncw · April 18, 2019, 12:19pm

You are using two different drive remotes though?

I think it time for a new issue on github - would one of you make one please?

calisro · April 18, 2019, 12:27pm

Yes. I sync the underlying contents of a primary crypt to alternate gdrives. I’ll open an issue and try to get more data for it as well.

calisro · April 18, 2019, 12:53pm

You can follow that issue for a resolution (hopefully )

bnr0723 · April 21, 2019, 8:21pm

I’m having a similar issue. However, I’m trying to copy Google Documents and other native formats from one remote to another, so the --disable copy flag would change the formats, which I don’t want. Otherwise I keep getting the same 404 errors as here.

Here is my post.

Leprechaun · May 5, 2019, 2:58am

I have a similar 404 problem with copying from two different Google accounts. I don’t need server side copying, I have a fast connection to pipe it though. I’m having trouble with syntax:

rclone --disable Copy -vv remote: remote:folder
rclone --disable copy -vv remote: remote:folder
rclone copy -vv --disable remote: remote:folder

Doesn’t work for me. I can’t find the documentation for it here:
https://rclone.org/commands/

Thanks for the help

Animosity022 · May 5, 2019, 3:09am

rclone copy -vv --disable copy remote: remote:folder

Leprechaun · May 5, 2019, 3:22am

Thanks! I originally tried something similar with no luck, I don’t remember what I was doing wrong. I don’t care, it’s working now.

durval · May 11, 2019, 6:11pm

Hello everyone,

Just to add more data about this bug: I'm seeing it here too, and it happens even when copying single files from a GoogleDrive remote to another.

As suggested by @calisro and confirmed by @igor_andreev, the workaround was to disable server-side copy via --disable copy. In my case, the source and destination remotes are owned by different users.

Hope this gets fixed soon, it's a real drag here.

Cheers,
-- Durval.

ncw · May 11, 2019, 6:59pm

I'm wondering if I should make the server side copy opt in (for different remotes) as calisro suggested on the issue as this has caused rather a lot of problems.

durval · May 11, 2019, 8:36pm

Hello @ncw,

I think it would be a good idea while the bug causing the issue isn't fixed... it cost me a few hours here trying to reduce a rather complex procedure to a single command capable of reproducing the problem, them searching the forum and finding it was already known... :-/

Cheers,
-- Durval

soy_titooo · May 15, 2019, 9:11am

I'm also having problems to do server side copies between two gdrive accounts.I never had that problem before till recently...

If I add the parameter "--disable copy" then I can upload again but I'm guessing that it would be much slower than server side copies...

Do you guys think it will be possible to fix it?

PS: I'm on rclone v1.47 just in case it matters

Animosity022 · May 15, 2019, 10:51am

There is already an issue logged if you read the thread.