Time to open file is a lot higher when using cache mode full

[random404]

I have a glusterfs storage cluster, mounted at /mnt/glusterfs. And I wanted to use rclone mount with the vfs cache so I can cache contents of the cluster locally, saving bandwidth on the cluster.

I noticed there is a huge performance drop in time to open files

My flags:

  --allow-other \
  --vfs-cache-max-age=168h \
  --vfs-cache-max-size=3T \
  --vfs-cache-mode=full \
  --cache-dir=/mnt/cache \
  --vfs-read-chunk-size=256K \
  --vfs-read-chunk-size-limit=8M \
  --buffer-size=256K \
  --no-modtime \
  --no-checksum \
  --dir-cache-time=72h \
  --timeout=10m \
  --umask=002 \
  --log-level=DEBUG \
  --log-file=/opt/rclone.log \
  --async-read=false \
  --rc \
  --rc-addr=localhost:5572 \

I tested with a starting chunk size of 1MB with buffer size of 1 MB too and found no noticeable difference in time to open files.

Compared to reading the same amount of data directly from the cluster:

Any ideas in how to fix this? Can we have a setting that create all the sparse files in advance, and maybe never clear them from cache

My rclone remote is just:

[glusterfs]
type = local

[ncw (Nick Craig-Wood)]

I can't see anything obviously wrong with your setup.

Can you attach a log please? Preferably from just before the dd starts to when it ends.

@random404

Any ideas in how to fix this? Can we have a setting that create all the sparse files in advance, and maybe never clear them from cache

I don't think the creation of the files is the problem here.

You could try pointing your mount to a local directory and see what the performance is like there.

[random404]

I have found the rclone mount is unusable, no matter the chunk size settings, even default.

As soon I start using the mount heavily nothing works and I don't see anything obviously wrong in the logs. I don't know why.

You can see the speeds directly from the cluster are much higher than the rclone mount with no active use...

I'm going to PM you my logs

[random404]

rclone/rclone/blob/2d7afe8690ece6285051143ddb9092fedca692d8/backend/local/local.go#L152

How can I change this setting to true on my mount?

[albertony]

@random404

How can I change this setting to true on my mount?

That has not been released yet, so you must run latest beta, then you can add command line option --local-no-preallocate.

[random404]

Thanks!!

But it didn't work... It went from 17 seconds to 14 seconds to read the first 32kb of file from cold cache...

Still unusable for me

Reading directly from the glusterfs mount takes less than 1 second to do the same