When listing the contents of a directory shared with me, some of its subdirectories are not listed, and I see no errors in the logs. If I login in the GDrive webUI using the same user that is authorized in the rclone remote and navigate into those folders, they become visible for rclone and I can list and copy them.
Run the command 'rclone version' and share the full output of the command.
~# rclone lsd pro:"/Premsa" -vv
2022/05/18 16:06:08 DEBUG : rclone: Version "v1.59.0-beta.6141.d206761d2.fix-drive-resourcekey" starting with parameters ["rclone" "lsd" "pro:/Premsa" "-vv"]
2022/05/18 16:06:08 DEBUG : Creating backend with remote "pro:/Premsa"
2022/05/18 16:06:08 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2022/05/18 16:06:08 DEBUG : pro: Loaded invalid token from config file - ignoring
2022/05/18 16:06:08 DEBUG : Saving config "token" in section "pro" of the config file
2022/05/18 16:06:08 DEBUG : pro: Saved new token in config file
2022/05/18 16:06:08 DEBUG : fs cache: renaming cache item "pro:/Premsa" to be canonical "pro:Premsa"
-1 2022-04-26 12:23:23 -1 BC
-1 2022-04-26 12:22:49 -1 Ca
2022/05/18 16:06:09 DEBUG : 6 go routines active
The directory listing should show a 3rd directory (I only realized because a user complained about it missing). I went to GDrive via web with the same user account that I authorized for this remote, opened the invisible directory and now I can list it and copy it with rclone.
I suspect these are from folders shared with a resourcekey in the URL.
I just added a fix to the latest beta so that you can actually download files.
So first can you try the beta to see if you can download files with it.
Was this directory from one which was originally shared with a link?
It might be that rclone needs to be using the resourcekey when listing the directory too - documentation is scant on the google drive API docs about this.
Not sure if the beta should be able to list (and copy) these directories before I navigate into them in the web UI, or only copy them after?
I downloaded the whole share via web UI and I'm looking at the diff between the outputs of the ~# find command on that downloaded tree and the one I get with the rclone beta (rclone-v1.59.0-beta.6154.fcec4bedb-linux-amd64).
I'm still finding some directories in the tree downloaded via the web UI that rclone didn't get. Is there any useful output I can provide if I find more cases of this?
The beta should be able to list the directory once you've opened it in the web. I don't know whether it will be able to list subdirectories of the main one though.
Try opening them in the web interface, then see if rclone copies them would be a good first step.
Yes, I can list and copy the contents of the directory after opening it via web, but subdirectories are not listed until I navigate into each one of them.
Yeah, that works.
Yup! The URL is something like: https://drive.google.com/drive/folders/0B2...TkU?resourcekey=0-S...8pw&usp=sharing
This should pick up the resource keys for the sub directories automatically.
Can you try rclone lsf -R pro:"/Premsa" to see if that works to find all the files.. If it doesn't try rclone lsf --disable ListR -R pro:"/Premsa".
This should also be able to list a drive without opening it in the web interface first. You'll need to supply the resourcekey as --drive-resource-key or in the config as resource_key.
Using --disable ListR I get a higher count than on the downloaded archive, which I guess can be due to duplicates.
I'll try to look into detail at these differences, but the web UI download renames many paths (e.g. those using special characters) which makes it difficult to use diff to compare them with the ones obtailed by rclone lsf.
I wasn't able to use the command line option, I get this error:
The --disable ListR looks to be the correct value.
I was concerned that ListR wouldn't work with resource keys as it lists multiple directories at once which means specifying multiple resource IDs. This should work but there have been a lot of corner cases in ListR.
What I suspect you are going to see is that whole directories are missing from the missing-in-without-listr. Assuming that is the case and that I constructed the headers correctly then this is probably a(nother) bug in the google search query handling. It might be better to disable ListR if resourcekeys are in use.