Cache files from google drive

is it possible to cache files from google drive somehow? so that the accesses go to the cache, say, on the nvme disk, or ram

My problem is that accessing google drive takes a long time to respond, I think creating a cache can speed it up, I made a mount point with --cache-dir but it was not filled with anything.
Can anyone help with this, I am ready to hire such a person.

rclone v1.57.0

  • os/version: ubuntu 20.04 (64 bit)
  • os/kernel: 5.4.0-99-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.17.2
  • go/linking: static
  • go/tags: none

Google Drive

hi,

when you posted there was a template of questions.
help us to help you and answer them.

rclone does not have a permanent cache.

can use rclone mount using a high value for --vfs-cache-max-age

or keep the files local and rclone sync/copy from local to gdrive.

when you first posted, there was a template of questions, the answers help us to help you.

rclone mount does not download files on its own.
the app accessing the mount does that.

to fill up the vfs file cache, need an app to download the files.
for example, can do a rclone md5sum on the mountpoint

as you are not providing the requested information, hard to share good advice.
please post
--- the redacted config file
--- the exact command, not just a snippet.

first, need to clean up those flags, understand what they do
--- remove --allow-non-empty, unless you are 1000% sure you need it.
--- --cache-tmp-upload-path does nothing, can remove that.
--- you have a bunch of duplicate flags, some with conflicting values

improve what?

cannot copy the vfs file cache to another computer.

on that server, run rclone copy/sync to download the files.

text removed by author

then run rclone mount on the server and it will have its own cache.

or have both servers share the same mount point /gdrive

text removed by author

text removed by author

the vfs file cache uses chunked reading.
rclone only downloads the chunks requested by whatever app is accessing.

as documented at https://rclone.org/commands/rclone_mount/#vfs-chunked-reading
"When rclone reads files from a remote it reads them in chunks. This means that rather than requesting the whole file rclone reads the chunk specified"

for example, if you have a 10GB video file in gdrive and use a media player to stream the first 10 seconds.
then rclone will only download that 10 seconds of the file, so the cache will contain only a small part of that 10GB file.

text removed by author

you cannot copy the vfs file cache from one computer to another. it is not designed for that.

text removed by author

maybe i am no understanding your full use-case

first, you wanted to cache gdrive files and i offered suggestions
then you wanted to copy that cache to another server, that is not possible and i offered workarounds.

sorry, at this point, not sure what you need help with?

text removed by author

text removed by author

rclone caches only the chunks of the file accessed by whatever app you are using.
if you have a 10GB file in gdrive, and only access 1MB, then rclone only downloads 1MB of data into the vfs file cache.
i offered a way to download the full file into the vfs file cache.

please let me know what is not clear?

text removed by author

https://forum.rclone.org/t/cache-files-from-google-drive/29557/7