About a week ago Google deleted all my data (150TB ), so my whole Plex library needs to be rebuilt. This time I am going to use local storage on my own server, combined with a friends server via NFS. At the moment my server only has the NFS mount on the remote server but I would like to also use local disk on my own server.
Not exactly sure what the best way to achieve this would be. Since I have been using gdrive for storage I am not using hard links, but I would like to start using it to avoid wasting storage for seeding. I have read about mergerfs, but I am not sure what the difference between rclone union and mergerfs/normal rclone mount is.
I do not care about redundancy really. In a perfect world I would just like to have a bunch of drives pooled together much like rclone union, but with the ability to hard link. If one of those drives crashes I lose everything stored on that particular drive but not the rest. Is that possible? Will the NFS mount cause issues with seeding compared to local storage? Both me and my friend have very stable, high bandwidth connections (Gbit+) but we are 7 time zones apart so the latency is around 90ms. I currently use vfs cache full but not sure if it's necessary with the new setup?
Both servers are running Linux and are connected via wireguard tunnel. Any tips/advice would be much appreciated!
I'm in a similar boat as I'm losing access to unlimited GDrive in a couple months. If you want to use hard links you'll need to use mergerfs as this is not supported by rclone's union.
Personally I plan to try out the following:
Build a NAS with 20TB drives and a 1TB SSD for OS/caching.
Use rclone union to merge all-1 of the 20TB drives. lfs will be used to minimize the number of drives being written to.
Use rclone mount with the union acting as the remote. vfs-cache-mode will be set to full with long max-age and write-back times so minimize spinning up the HDs.
Run SnapRAID once a day to have single parity fault tolerance.
Reasons I'm trying rclone before mergerfs:
I want to minimize spinning up the HDs as much as possible. You can setup tiered caching with mergerfs but if someone attempts to read a file that's not currently in the cache mergerfs will first transfer it to the cache before the user can access it. For 10+GB files that's a lengthy delay for the user. Rclone instead performs caches on a bit level, so there's no delay for the user.
I prefer to use docker when possible, and it's much easier to deploy rclone than mergerfs.
It'll be a lot more convenient if I ever decide to setup remote storage in the future for a subset of my data.
Look, moving away from the Rclone tool for anyone interested in storing movie libraries and TV shows locally, I would recommend Nextcloud in conjunction with Plex, as both support aggregating multiple disks and mapping them as external storage, so For example, let's imagine that we have 5 HD's of 20 TB and that each of them has specific content. You can simply map it as follows in Plex Biblioteca Filmes /mnt/HD1 /mnt/HD2 /mnt/HD3/Filmes and for Programs TV would look like this /mnt/HD3/TV /mnt/HD4 /mnt/HD5 and then the mount points would be mapped within Nextcloud or the other suggestion is to do a Raid
My Setup here is the following: I have 30 TB between TV Shows and Movies, just to not spend a lot of money, so what I did
I purchased 3 Microsoft 365 subscriptions totaling 18 TB (1 TB per account)
So it looks like this: TV programs with better quality are on OneDrive, where I set up accounts with union in /mnt/TV/Full
A 6 TB HD Connected 24/7 to my Server, where I separated 4 TB for some movies and TV shows and the other 2 TB for Family Archives.
A second TB HD that remains for smaller versions and conversions of PLEX including the Rclone cache and with this both the 6 TB drives and OneDrive accounts are mapped in PLEX, if something fails, the person can still play the summer's content web.
And finally, I keep a copy of all content stored on other hard drives that are saved and turned off. If a drive fails, I can recover the content or even forget to pay for OneDrive, or Microsoft deletes my files and I just go there and buy it again and upload it.
), so my whole Plex library needs to be rebuilt. This time I am going to use local storage on my own server, combined with a friends server via NFS. At the moment my server only has the NFS mount on the remote server but I would like to also use local disk on my own server.