I have --fast-list in my copy commands, but just noticed today that it seems to be giving me an error:
Error 400: Invalid Value
if I remove --fast-list the error goes away
so does anyone know if this been deprecated from the google drive api?
thanks
rclone version shows?
Can you run the same command with -vv and share the output?
If that doesn't show much, can you run the same command with --dump headers as well?
rclone version is 1.48 (I updated from 1.47 in case that was the problem)
-vv was not very illuminating 
I did try.
I will try with --dump headers and see what happens
I tried with --dump headers and the only times I see error 400 in the output is in blocks:
019-07-17 20:36:18 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2019-07-17 20:36:18 ERROR : : error reading destination directory: couldn't list directory: googleapi: Error 400: Invalid Value, invalid
2019-07-17 20:36:18 INFO : Google drive root 'plex': Waiting for checks to finish
2019-07-17 20:36:18 INFO : Google drive root 'plex': Waiting for transfers to finish
2019-07-17 20:36:18 ERROR : Attempt 1/3 failed with 1 errors and: couldn't list directory: googleapi: Error 400: Invalid Value, invalid
2019-07-17 20:36:18 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
and it does the same thing on both an encrypted folder and a non encrypted folder
If it is specific folders on my drive that are causing the issues, not sure how I would find them.
Not sure I want post/share the whole of the --dump headers output
I can confirm the same behavior happens here even when using rclone ls : on a Google Drive mount. --fast-list was not invoked, so I'm not sure that is the culprit. I am also using rclone 1.48.0. I have run the command above both in linux and Window 10. The list will stop parsing at random different places with the truncated error:
Failed to ls: couldn't list directory: googleapi: Error 400: Invalid Value, invalid
The debug-header before the stop point said:
2019/07/18 03:54:32 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2019/07/18 03:54:32 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2019/07/18 03:54:32 DEBUG : HTTP RESPONSE (req 0xc000101500)
2019/07/18 03:54:32 DEBUG : HTTP/1.1 200 OK
Transfer-Encoding: chunked
Alt-Svc: quic=":443"; ma=2592000; v="46,43,39"
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Wed, 17 Jul 2019 20:54:31 GMT
Expires: Wed, 17 Jul 2019 20:54:31 GMT
Server: GSE
Vary: Origin
Vary: X-Origin
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block2019/07/18 03:54:32 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2019/07/18 03:54:32 Failed to ls: couldn't list directory: googleapi: Error 400: Invalid Value, invalid
The error is reproducible over and over again, but the list will stop parsing at different points each time.
Can you share what you are actually running ?
Can you share what you are actually running?
rclone ls Media: -vv --dump-headers
Yep, just tried a basic
rclone ls -vv --dump headers gDrive:folder
and saw the same behavior.
the first 2 times it stopped with the error at different points, the third time it completed successfully, so am thinking it's some sort of google glitch
Can you grab the full log?
The full log logs looks like:
2019/07/17 17:09:17 DEBUG : rclone: Version "v1.48.0" starting with parameters ["rclone" "lsd" "GD:" "-vv"]
2019/07/17 17:09:17 DEBUG : Using config file from "/opt/rclone/rclone.conf"
-1 2019-07-09 13:14:42 -1 gemini
-1 2017-06-09 09:59:43 -1 media
-1 2019-07-09 15:28:42 -1 testing
2019/07/17 17:09:17 DEBUG : 5 go routines active
2019/07/17 17:09:17 DEBUG : rclone: Version "v1.48.0" finishing with parameters ["rclone" "lsd" "GD:" "-vv"]
Can you grab the full log and post that?
I sure will when the error happens again. As of now, that error isn't showing up. I'm sure it's something on Google's end as the coders change whatever it is they are doing.
Your log example is from the lsd command not the ls command, I would really rather not post the whole of the ls log as that contains the content of my drive
I just gave an example if you want to write to a log file, that might show something.
2019/07/17 17:10:47 DEBUG : rclone: Version "v1.48.0" starting with parameters ["rclone" "ls" "-vv" "--fast-list" "GD:" "--log-file" "/tmp/fulllist.log"]
2019/07/17 17:10:47 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2019/07/17 17:11:10 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=286927808882, userRateLimitExceeded)
2019/07/17 17:11:10 DEBUG : pacer: Rate limited, increasing sleep to 1.158673967s
2019/07/17 17:11:10 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=286927808882, userRateLimitExceeded)
2019/07/17 17:11:10 DEBUG : pacer: Rate limited, increasing sleep to 2.483521162s
2019/07/17 17:11:11 DEBUG : pacer: Reducing sleep to 0s
2019/07/17 17:11:28 DEBUG : 21 go routines active
2019/07/17 17:11:28 DEBUG : rclone: Version "v1.48.0" finishing with parameters ["rclone" "ls" "-vv" "--fast-list" "GD:" "--log-file" "/tmp/fulllist.log"]
There's another example of just the log if you don't want to share contents. Log files are key in figuring out what the issue is. If you don't want to post them, you can share them via PM or send them to ncw if you want him to take a look.
--fast-list not needed. See log below.
2019/07/18 04:22:47 DEBUG : rclone: Version "v1.48.0" starting with parameters ["rclone" "ls" "GDrive:" "--config" "D:\\rclone\\.rclone.conf" "-vv" "--log-file=D:\\rclone\\error.log"]
2019/07/18 04:22:47 DEBUG : Using config file from "D:\\rclone\\.rclone.conf"
2019/07/18 04:22:49 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=XXXXXXXXXXXX, userRateLimitExceeded)
2019/07/18 04:22:49 DEBUG : pacer: Rate limited, increasing sleep to 1.004139897s
2019/07/18 04:22:49 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=XXXXXXXXXXXX, userRateLimitExceeded)
2019/07/18 04:22:49 DEBUG : pacer: Rate limited, increasing sleep to 2.583207438s
2019/07/18 04:22:49 DEBUG : pacer: Reducing sleep to 0s
2019/07/18 04:23:07 Failed to ls: couldn't list directory: googleapi: Error 400: Invalid Value, invalid
You can use three backtics before and after, which is ` thing.
example
ok thanks, so writing to a log file still does not give much of a clue...
2019/07/17 21:44:59 DEBUG : rclone: Version "v1.48.0" starting with parameters ["rclone" "ls" "-vv" "--fast-list" "gDrive:plex" "--log-file" "fulllist2.log"]
2019/07/17 21:44:59 DEBUG : Using config file from "/path/to/rclone.conf"
2019/07/17 21:44:59 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=XXXXXXXXXX, userRateLimitExceeded)
2019/07/17 21:44:59 DEBUG : pacer: Rate limited, increasing sleep to 1.296368801s
2019/07/17 21:44:59 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=XXXXXXXXXX, userRateLimitExceeded)
2019/07/17 21:44:59 DEBUG : pacer: Rate limited, increasing sleep to 2.22033069s
2019/07/17 21:45:00 DEBUG : pacer: Reducing sleep to 0s
2019/07/17 21:45:01 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=XXXXXXXXXX, userRateLimitExceeded)
2019/07/17 21:45:01 DEBUG : pacer: Rate limited, increasing sleep to 1.712570592s
2019/07/17 21:45:01 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=XXXXXXXXXX, userRateLimitExceeded)
2019/07/17 21:45:01 DEBUG : pacer: Rate limited, increasing sleep to 2.49674609s
2019/07/17 21:45:01 DEBUG : pacer: Reducing sleep to 0s
... repeated a few more times and then...
2019/07/17 21:45:58 Failed to ls: couldn't list directory: googleapi: Error 400: Invalid Value, invalid
And this morning it all seems to have returned to normal, the error message seems to have gone away.
So I am guessing google were fiddling around with something.
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.