Rclone GDrive error

Help and Support
1

What is the problem you are having with rclone?

Getting this error everytime the command runs. Is there something that is needed to be done to this folder?

2020/07/19 07:48:20 NOTICE: Google drive root 'Encrypted/8nak3756llo68qkpg06nt70mg0': Disabling ListR to work around bug in drive as multi listing (3) returned no entries
2020/07/19 07:48:20 NOTICE: Google drive root 'Encrypted/8nak3756llo68qkpg06nt70mg0': Re-enabling ListR as previous detection was in error

What is your rclone version (output from rclone version)

1.52.2

Which OS you are using and how many bits (eg Windows 7, 64 bit)

It is a docker image (RHEL7)

Which cloud storage system are you using? (eg Google Drive)

GDrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

--stats-file-name-length 100 --size-only --transfers=1 --fast-list --bwlimit 10M --stats 90s --log-file /logs/movies_backup.log -v

The rclone config contents with secrets removed.

[Gdrive]
type = drive
scope = drive
client_id = 
client_secret = 
token = {"access_token":"","token_type":"Bearer","refresh_token":"","expiry":""}
root_folder_id = 

[Crypt]
type = crypt
remote = Gdrive:/Encrypted
filename_encryption = standard
password = 
password2 =

A log from the command with the -vv flag

2020/07/19 07:48:20 NOTICE: Google drive root 'Encrypted/8nak3756llo68qkpg06nt70mg0': Disabling ListR to work around bug in drive as multi listing (3) returned no entries
2020/07/19 07:48:20 NOTICE: Google drive root 'Encrypted/8nak3756llo68qkpg06nt70mg0': Re-enabling ListR as previous detection was in error
2

hi,

  • that sure is strange log entries.
  • can you post the command?
  • did this start with v1.52.2?
  • can you run and post the command with debug log?
3

Thanks for the quick response -- I am already running this with verbose logging as you can see below. The is the log entry that gets spammed when the rclone command is run in the docker instance.

This is the command

--stats-file-name-length 100 --size-only --transfers=1 --fast-list --bwlimit 10M --stats 90s --log-file /logs/movies_backup.log -v

Those are all the options that are passed except the paths and the "rclone" command.

From what I can tell this is new...

4

it can be confusing, -v versus -vv

when you posted, you were asked for

A log from the command with the -vv flag

https://rclone.org/docs/#log-level-level
INFO is equivalent to -v
DEBUG is equivalent to -vv

yes, can you post that.
the paths should not reveal anything secret

5

I'll add the -vv to it

That is essentially the command is my point. I can show you all the variables I pass to it, but since it's a docker it does not have a generic rclone command that is listed:

This is the docker container i'm using: https://hub.docker.com/r/pfidr/rclone

Here are all the docker environment variable I pass to it:

|SYNC_SRC|/source_folder|
| --- | --- |
|SYNC_DEST|Crypt:/Movies/|
|RCLONE_CMD|copy|
|FORCE_SYNC|1|
|CRON|45 * * * *|
|SYNC_OPTS|--stats-file-name-length 100 --size-only --transfers=1 --fast-list --bwlimit 10M --stats 90s --log-file /logs/movies_backup.log -v|
|UID|0|
|GID|0|
|TZ|America/Los_Angeles|
|PATH|/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin|
|SYNC_OPTS_EVAL||
|SYNC_ONCE||
|RCLONE_DIR_CMD|ls|
|RCLONE_DIR_CMD_DEPTH|-1|
|RCLONE_DIR_CHECK_SKIP||
|RCLONE_OPTS|--config /config/rclone.conf|
|OUTPUT_LOG||
|ROTATE_LOG||
|CRON_ABORT||
|CHECK_URL||
|FAIL_URL|

edit: the command would be similar to this:

rclone copy --stats-file-name-length 100 --size-only --transfers=1 --fast-list --bwlimit 10M --stats 90s --log-file /logs/movies_backup.log -v /source_folder Crypt:/Movies/
6

Depending if you just moved a bunch of stuff, Google Drive sometimes take some time to catch up. It doesn't seem to be a team drive though as that's normally where I see it.

fast-list does some magical indexing on the backside to speed things up and if it detects an error, it just does a regular listing so I don't think that should be an issue and should go away on its own over time.

@ncw can chime in as well.

7

This message is caused by this code

If you want to see the full story then check that out and the issues linked from it.

In short though

  • There is a bug in Google Drive listing when using --fast-list or one of the commands which enables it automatically (like rclone ls or rclone lsf -R)
  • Rclone attempts to work around this by switching back to normal listing mode if it detects listings from multiple directories being empty
  • This detection isn't 100% accurate though so sometimes it switches back again.

If you have multiple empty directories in the same directory you'll likely trigger this message.

It is harmless.

I maybe should downgrade it from a NOTICE to a DEBUG - thoughts?

1 Like
8

That's helpful as moving it to DEBUG should be good.

9

Agreed that moving to NOTICE would be better :wink: -- Thanks for the information. I don't have a lot of empty directory but I do have a LOT of directories, I think over 1k in that folder. So maybe it's having a tough time getting them all.

As long as it's not effecting anything I don't mind but it would clean up the NOTICE logs which would be nice.

10

OK I've moved the messages down to DEBUG :slight_smile:

This will be in the latest beta in 15-30 mins.

I'll put it in 1.52.3 or 1.53 whichever comes soonest!

11

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.