Rclone mount cache requests Gdrive

Help and Support
1

What is the problem you are having with rclone?

Hello,
I have got specific case. I run rclone on my Linux. Inside of my google drive is for example 30 files with 100gb size each of them. I have got application which frequently try to check info about this files but not download all 100gb. Is it possible to cache info about files?

Currently I try in this way and it works in half :slight_smile: :

rclone mount --allow-other --dir-cache-time 720h --poll-interval 15s --cache-dir=/home/work/cache_test/ --vfs-cache-mode full --vfs-cache-max-size 20G --vfs-cache-max-age 99h --transfers 9999999 --vfs-cache-poll-interval 15m folder: /home/work/folder --daemon

The problem is that, that after first request one of file, I need to wait about 20mins to finish. Then it is cached and new requests works very fast like I want. Is it possible to cut this 20min? I don't know why it takes so long. If I have for example 30 files then I need to wait 30 x 20min = 600min

Without vfs flags and without cache option one request takes about 60sec.

Maybe rclone can't do that and I need to use a different tool like squid?

What is your rclone version (output from rclone version)

rclone v1.50.2

  • os/arch: linux/amd64
  • go version: go1.13.6

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Linux Ubuntu 20.04 LTS

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount --allow-other --dir-cache-time 720h --poll-interval 15s --cache-dir=/home/work/cache_test/ --vfs-cache-mode full --vfs-cache-max-size 20G --vfs-cache-max-age 99h --transfers 9999999 --vfs-cache-poll-interval 15m folder: /home/work/folder --daemon

The rclone config contents with secrets removed.

[test]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = {"access_token":"XXX","token_type":"Bearer","refresh_token":"XXXX","expiry":"XXXX"}
team_drive = XXXX
2

Missed all the details in the help template that would help answer that question :frowning:

So it's super hard to answer you with all that juicy information.

3

Sorry. I just edited my first post. Should be better.

4

You have an old version so vfs cache mode full downloads the full file.

You need to update to the latest to use spare files.

5

After app update should I keep all current flags? Should I modify something?

6

Lots of those flags are really specific to what you want to do / what you expect to happen.

Probably give it a whirl and if there is a specific question, feel free to ask.

8

Is it possible to cache only metadata or list of files?

9

That's done by the dir-cache-time parameter. You can set that very high on Google Drive as it's a polling remote.

10

Can you help a little? Because this cache don't work for me :frowning:
rclone mount --allow-other --dir-cache-time 24h --cache-dir=/home/work/cache_test/ --no-modtime test: /home/work/test/ --daemon

11

What isn't working?

12

I would like to cache only dirs. No files. I think that it is impossible. I need use --vfs-cache-mode full flag if I want to cache dirs. It also cache files.
Currently my config looks like this:
rclone mount --allow-other --vfs-cache-mode full --vfs-cache-max-age 8640m --vfs-cache-max-size 10G --dir-cache-time 24h --cache-dir=/home/work/cache_test/ --no-modtime test: /home/work/test --daemon

13

If you use dir-cache-time, it keeps the directory and file structure. There's really no reason to not have both as it's just metdata.

This is has not much really do with the prior flag, but it's for caching files locally.

14

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.