I use zfs to pool together nvme disks into a single folder for rclone to use vfs cache.
I was reading this blog Tuning recordsize in OpenZFS | Klara Inc
Does anyone knows what is the ideal recordsize for zfs for maximum performance with rclone?
And what you will use it for? To store and stream video files? Or to access heavily used Oracle database instance...
I think you have to provide much more detailed requirement for somebody to advice.
IMO given FUSE + VFS overhead penalty and that you use NVME drives you should not see any performance issues due to ZFS record size. But I am curious myself what your experience is gonna be.
Sometimes in cases like this the best what you can do is to test different values under your expected workload. And choose one which the best fits what you want - the highest IOPS? the lowest write amplification? highest sustained throughput?
most of the time is to store and stream video files yeah.
I was just more curious if I should make the recordsize match the fuse to avoid issues
1 Like
1MB record size with LZ4 compression is what I would start with.
Yeah if we were storing the files directly.... but is rclone that is going to write to it using fuse and what not so i get lost lol !
and also the cache folder also has a folder for metadata with lots of small files.
and I don't even know how the sparse files goes with this too
1 Like
rethinking about it, the remote is the one storing the large files, the zfs is for the vfs cache so should be optimal for rclone writing patterns / sizes
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.