Linux NFS Server with Rclone and local disk for cache

Been thinking for a while to set up a server with 5 ssd in raid 0.
Mount rclone and put the cache directory on the ssd's then share it on nfs for other servers.

Point is that i want speed on this share for upload and download from the share. and if its not in the cache it will bee downloaded from the cloud storage.

Anyone running a setup like this? There will be lots of activity on this share.

Tried to make a nfs share now.

got this message:
okt. 23 10:58:21 nfs-server rpc.mountd[1466]: authenticated mount request from 192.168.1.108:777 for /gdrive/pbs (/gdrive/pbs)
okt. 23 10:58:21 nfs-server rpc.mountd[1466]: Cannot export /gdrive/pbs, possibly unsupported filesystem or fsid= required

See this page for adding an fsid / using nfsv4 which should hopefully solve the problem.

So this is my export file:

/gdrive/backup 192.168.1.10(rw,sync,no_root_squash,no_all_squash,fsid=0)

And i cant write to it.

root@vhost01:/mnt/pve/backup/test# touch test
touch: cannot touch 'test': Input/output error

Have anyone tested something like this before?

Did you put --allow-other and possibly --allow-root on the rclone mount?

yes this is my mount string:

rclone mount gcrypt: /gdrive --allow-other --allow-root --dir-cache-time 720h --poll-interval 15s --cache-dir=/cache --vfs-cache-mode full --vfs-cache-max-size 10G --vfs-cache-max-age 336h &

think this is a known problem:

Or are there anyone using a setup like this?

Yes you are right, and I don't think we've figured it out yet.

Digging into the s3ql source might be helpful to figure out what the --nfs workarounds do.

OK, thanks. I can to some digging to se if i figure out something. In the mean time is there somewhere i can subscribe to get updates on this if its fixed?

If you subscribe to the GitHub issue above, you'll get notification.

thanks to this fix https://beta.rclone.org/branch/fix-2115-nfs/v1.54.0-beta.4869.bfcd4113c.fix-2115-nfs/
NFS Share is now possible and seems to be working very well.

Also is there any suggestion of how this should be mounted?

My current mount:

rclone mount gcrypt: /gdrive --allow-other --dir-cache-time 720h --poll-interval 15s --cache-dir=/cache --vfs-cache-mode full --vfs-cache-max-size 1024G --vfs-cache-max-age 3360h --write-back-cache --vfs-write-wait 60m &
Thougts:

Some of my thoughts here are:
vfs-cache-max-age should be forever and when cache disk is full the least used file should be removed for creating of new files. this because of saving bandwidth and increase of speed.

write back cache and vfs cache wait is for when adding files should go very fast and rclone can work quite in background uploading files.

Gladly to for input and thoughts on this.

I've merged the fix for the NFS problems to the latest beta now.

This is too big --vfs-write-wait 60m - you should probably leave it at the default

--write-back-cache controls kernel caching rather than anything else. I'm not sure how important it is.

The other settings look fine :slight_smile:

Been using this setup for a while now and have to say it works great if other need to use this by nfs server. I have 10Gbit network connection and seen it go up to 4gbit some times. So it works perfect.

1 Like