Mount with slow performance on selfhosted Garage Instance

Help and Support
1

What is the problem you are having with rclone?

Directory Listing and File listing are quite slow, even though I thought having VFS on Full should cache all folders and files. Is that correect?

Here is what I have checked already:

  • I played around with the config and used different attributes for dir-cache-time and vfs-* options
  • I checked the /home/xxx/SX135cache and files transfered are stored there and the 100GB max size is beeing used, but not for Folder/File List caching, just for the actual files themself.

The main Problem is, starting a media scan in plex can take very very long, not only the first time, but every single time. And whenever I restart the rclone service, I also have to restart plex, otherwise it will not find any folders at all.

I'm realy a bit confused, since the almost identical configuration worked for both Gdrive and Dropbox in the past.

What am I missing, what can I optimize?

Storage system

I have a 133TB Storage Server (Hetzner SX135) with garage as S3 compatible api running on it.

'rclone version'

rclone v1.67.0
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 5.15.0-122-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.22.4
- go/linking: static
- go/tags: none

'rclone config redacted'

[SX135]
type = s3
provider = Other
env_auth = false
access_key_id = XXX
secret_access_key = XXX
endpoint = http://10.13.37.10:3900
region = XXX
acl = private

[SX135CRYPT]
type = crypt
remote = SX135:media
password = XXX
password2 = XXX

'mount service'

[Unit]
Description=RClone Service SX135
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
Environment=RCLONE_CONFIG=/home/xxx/.config/rclone/rclone.conf
RestartSec=5
ExecStart=/usr/bin/rclone mount SX135CRYPT: /mnt/storage \
--allow-other \
--dir-cache-time 9999h \
--log-file /home/xxx/services/rclone/logs/rclone-XS135.log \
--log-level INFO \
--umask 002 \
--rc \
--rc-addr 127.0.0.1:5576 \
--rc-no-auth \
--cache-dir=/home/xxx/SX135cache \
--vfs-cache-mode full \
--vfs-cache-max-size 100G \
--vfs-fast-fingerprint \
--vfs-cache-max-age 9999h \
--dir-cache-time 5s \
--tpslimit 12 \
--tpslimit-burst 0
ExecStop=/bin/fusermount3 -uz /mnt/storage
ExecStartPost=/usr/bin/rclone rc vfs/refresh recursive=true --url 127.0.0.1:5576 _async=true
Restart=on-failure
User=xxx
Group=xxx

[Install]
WantedBy=multi-user.target
2

hi,

that is expected behavior.
check out my summary of the two vfs caches

well, you have duplicate flag. i suggest --dir-cache-time 9999h
and should use a debug log, to see what rclone is doing.

i suggest removing ExecStartPost and use --vfs-refresh

why use that??

should update rclone by running rclone selfupdate

3

Thank you!

  1. I remove the second --dir-cache-time
  2. I removed the ExecStartPost
  3. I added --vfs-refresh
  4. I removed tps*
  5. rclone updated to the latest version

I also finally understood the differences between the caches: thanks! :wink:

I'll see how it behaves and will give a short update later on.

1 Like
4

Just to follow up: all is good :wink:

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.