I’ve been working quietly away, mainly posting on Slack. I have used and tested both rclone / acdcli with UnionFS and OverlayFS. I’ve come to the conclusion that rclone mount + crypt with local storage and ACD using OverlayFS to be the best performing setup. I’ve created a custom complied kernel v3.18.48 and v4.4.52 from source to modify the FUSE parameters in use by rclone mount to further increase performance.
Server I’m using is a online.net dedibox lt 2016 running ESXi with Debian 8, custom 4.4.52_1 kernel (600GB local SSD, 24GB RAM, 8 CPUs).
Amazon: EU
My location UK.
Method:
Download locally and retain until disk is 80% full then upload, pausing NZBGet while uploading and also only uploading during the hours of 01:00 to 15:00. NZBget is paused for downloads during 15:00 to 01:00 to ensure smoother playback.
I use OverlayFS to merge ACD and local storage.
Plex will load from 2-5 seconds with no buffering or stuttering. I’ve disabled all scanning from Plex. Sonarr/Radarr update Plex on download.
Any data to share about performance between OverlayFS and UnionFS? I’ve been using UnionFS for quite a long time and have been overall pretty happy but I have had the inclination to look at OverlayFS.
Unfortunately not, I’ve done this during the night looking after my newborn so logs were missed during testing. Overlay should give native disk performance based on kernels changelog, v4.4.52 has improvements as this is what docker overlay2 recommends and they have fed into fixes etc.
Thank you for reporting your findings.
Can you tell us how much of a difference the kernel options make? And did you only change VM_MAX_READAHEAD to 1024 and FUSE_MAX_PAGES_PER_REQ to 256?
Did you ever use encFS? If so why did you switch to crypt?
What is the reasoning behind using these parameters when moving local to acd: --exclude partial~ --acd-upload-wait-per-gb=10s?
Why aren’t you using max-readahead and acd-templink-threshold options for the crypt mount?
Also, how do handle disable full library scans via Sonarr and Radarr? Do you disable them?
For me without the kernel changes I was unable to directplay on any device, with these changes I can.
Yes i’ve only changed those two parameters for now but will look at other kernel tuning for network / filesystem / memory.
I did use encFS however it was awfully slow on writes and had massive diskio which made transfers / uploads really slow.
excluding partial files as we don’t want them in the cloud, upload-wait-per-gb is to reduce the overall time we are waiting for rclone to verify the files are in the cloud.
max-readahead doesn’t work well on unionfs and not required for overlayfs, we can change it for rclone mount now as we have increase the vm_max_readahead.
You can disable library scans from within Plex. Sonarr/Raddar update plex based on the file location.
Ajki, this is true if you want soley a cloud solution but some people like to have a local buffer for newer files.
I mostly have this working though picked up on a few problems along the way (I posted on reddit the issues I’d had getting the Rclone mounts the correct way around), I’ve also added an additional variable and flags to the Rclone functions to point directly to my rclone.conf. The exclude flag doesn’t actually work in the scripts if you run this command separately in a terminal, it needs a wildcard at the beginning.( e.g. $rclone --config=$RCLONECONF --verbose move --exclude “*partial~”).
The final piece of the puzzle I’m stuck on is getting this to work correctly with OverlayFS, after a move from Rclone the space isn’t actually freed up on the mount. Once the upload script has finished df -h returns the same disk space usage but when I poke around the .local upper directory files have been removed and the directories appear to be empty.
Anybody else have experience with this?
EDIT: I forgot to mention, the space is freed up after I complete a remount of the overlay mount
I think the issue is with overlayfs, any actions on the upper folder seem to need to happen through the mount rather then directly in the upper folder. I don’t know any so I’m solutions to this, but if you run rclone move on the upper folder, it will move the files but they won’t be cleared on the overlay until the overlay is restarted