Google Drive Query Feature

Hi All,

I'm a long time rclone user but switched to using prasmussen/gdrive for certain tasks a couple of years ago. The key benefit it had over rclone was searching for files based on a glob pattern, when there are a lot of files in the parent folder. This is a common task in my company as we have a flat binary artefact repository stored in Google Drive. Using rclone's filtering is very slow in this situation, because it filters on the client side, whereas gdrive sends a search query to Google and only gets matching responses back.

Unfortunately gdrive has been abandoned and although there is a successor project, it's not ready for production use and, in any case, I'd prefer to improve rclone's handling of this so we can use one tool for all our remote file access tasks.

Looking at the backend code for rclone list drive: there is an unused title argument. It would be good to know the story behind that, it almost does what we need, except that it doesn't support globs because there is an additional check on line 1140 that at least the file name stem must match the title. AFAICS the calling code always passes in an empty string so this code path is not used.

To get this working, I implemented a new backend command rlcone backend query drive: which accepts a query in Google Drive's native syntax, just like the gdrive tool does. I've submitted this as a pull request:

It would be great to get some feedback on this, it's my first time contributing to rclone so if there's a better way to achieve what we need, please let me know.

Thanks,

J-P

1 Like

It is possible to push the filtering down into the backend. Backends need to set the FilterAware feature flag. In fact you'll see drive does this already to implement --min-age and --max-age. You could extend this functionality to implement --include and --exclude. However rclone's filters are a lot more powerful than Google's so translating all the rules might be hard.

The title parameter is used to find a specific file in the FindLeaf function.

That sounds like a great thing to have even if we decide to do something else later.

I'll comment further on this on the PR.

Thanks for your feedback on this feature. I've pushed some changes to the PR, could you please take a look and let me know if this is ready to merge? I'm hoping to spread use of the feature to other teams in my company and would prefer us to be using the official release than my fork.

1 Like

I've merged this to master now which means it will be in the latest beta in 15-30 minutes and released in v1.67

Thank you :slight_smile:

1 Like