Best option for mounting immutable source on Linux

Help and Support
1

We are using LakeFS Server, that provide S3 compatible API and Azure Blob Storage.
We are using rclone mount to access to those data store on our Linux servers
LakeFS data is immutable (with protected branch): file don't get changed, created, deleted. For our Azure blob storage, because of the way we upload the data, it is also immutable like for Lakefs. If a file changes, that an issue from our upload end, which is out of scope of this topic

I would like to get the best performance of caching system knowing the above. I had read the docs and forum and found those options that I think it can be useful:
--attr-timeout 52w: no more kernel call to rclone once file is cached once ?
--dir-cache-time 52w: no more cache invalidation for directories ?
--vfs-fast-fingerprint: will this remove all fingreprinting ? I should not need fingreprinting right ?
--vfs-cache-mode full
--vfs-cache-max-age 52w: no more cache eviction ?
--no-checksum --no-modtime: do I need those with --vfs-fast-fingerprint ?
--read-only: good to have as the system is immutable anyway ?
--transfers 64: useful for parallel file read ?

Is there any option that I missed ?
Thank you for your help.

2

Looks good. Small adjustments:

indeed no need for this

no need - it only applies to uploads

I would add:

--vfs-cache-max-size SizeSuffix      Max total size of objects in the cache (default off)

to be in control of cache size - otherwise cache will potentially consume all your free disk space

To improve performance you could also optionally warm cache after mounting using remote interface (rc).

e.g. in systemd service:

[Unit]
Description=rclone mount

[Service]
Type=notify
Environment=RCLONE_CONFIG=/path/to/rclone.conf
RestartSec=5
ExecStart=/usr/bin/rclone mount src mountpoint \
-- all other flags \
--rc \
--rc-addr 127.0.0.1:5572 \
--rc-no-auth

ExecStop=/bin/fusermount3 -uz mountpoint
ExecStartPost=/usr/bin/rclone rc vfs/refresh recursive=true --url 127.0.0.1:5572 _async=true
Restart=on-failure
User=user
Group=group

[Install]
WantedBy=multi-user.target
1 Like
3

In your example above, you run rclone mount with --rc which make rclone start as a server and wait for command.
How do you run rclone without any operation, just as a server to wait for remote command ?

4

Oh found it : rcd

1 Like
5
6

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.