Fast searching cache-backend question


#1

@ncw

how to make rclone mount with cache good for fast searching?
What settings are best to make rclone cache-backend database quickly select from milions of files?

For example if i have on gdrive 6TB data, more than 1000000 of files and folders.
I would like to use rclone as quick indexed database to show me where is a file which i am searching, using only locally builded cache-backend database, not checking if file really exists on remote.

thanks.


#2

Do you want to search the file names or the file contents?


#3

file and folder names only.
so it would act like local database, and checking if file exist only on read or write access.
And if file is not there - throw some error and delete from database or initiate current folder cache/db update


#4

If you want to find a file with “hello” in its name you can use

rclone lsf --include "*hello*" remote:path

You can use --ignore-case too.

That will work on a cache backend too and faster since the cache should have the metadata loaded already.


#5

thanks i knew about this shell solution,
i was thinking about searching in mount folder.
Is it even possible?

Like using third party app, explorer in windows of file manager on android,
to search in mounted folder -> rclone would simulate existence of files in some mounted folder based on chached-backend.
Or it is fuse site and impossible and always will be slower?

of maybe possible to do using vfs?

currently i am using fx file manager on android, it has his own index .db file (sqlite) so searching is very fast, but asked out of curiosity for other file managers - which doesnt update own index or dont have it.


#6

If you’re going to search on a mount you need fuse and thats trouble on android without root. Even with root its becoming problematic. But if you’re simply going to execute rclone searches you can do that with the command line (or build an app to pass to rclone).