Hi there,
as many of you, I have been looking for a new storage solution and I wasn't really happy with the pricing of any of the cloud storage options, so I decided to build my own storage server, which has now been running perfectly for a couple of weeks. As some of you have expressed interest to follow my project, I would like to share my experiences here. There is quite a lot I have to share, so I will divide this into at least two parts.
Let's first get a quick overview of my previous setup, which was working perfectly for the last couple of years: I have a big collection of large files on a google team drive, which I had mounted with rclone on a dedicated server. The server was mainly used to stream the files over the internet. It is also powerful enough to host the occasional CPU-heavy game server (e.g. for satisfactory) and it benefits a lot from Intel's Quicksync, so an Intel CPU with integrated graphics is one of the requirements.
Recently, my team drive has become read-only, so I needed a new storage solution for new files (4 - 6 TB / month). Existing files can stay on the teamdrive for as long as possible, so don't worry, I will still be using rclone :). Everything I store can be found all over the internet and is easily replacable, so I do not really care about data integrity or backups. For my particular use case, I decided that it would be worth it to build my own storage server and colocate at a datacenter. This first part is intended to explain how I arrived at that decision by comparing the cost between different cloud storage options and colocated storage including the hardware that requires. In later parts, I will share more details about the build and any experiences that may help anyone planning a similar project.
Part I: Cost for Cloud vs. Local Storage
Cloud storage
The cheapest trustworthy cloud-storage options that I found were either IDrive e2 or Hetzner storage boxes:
IDrive e2 offers S3 compatible storage for 4$ / TB / month on a flexible plan, or for only 1.67$ / TB / month on a yearly plan.
Hetzner storage boxes vary in price between 3.81€ / TB / month for the smallest box and 2.42€ / TB / month for the largest box (20TB). Boxes can be extended whenever needed, so to save costs I could start with a small box, then extend it, then add another small box, extend it again etc. However, to keep it practical I calculate only with 10 TB and 20TB boxes, as with smaller boxes it would become too cumbersome to manage.
Let's compare the cumulative cost between the options during the first two years. I calculate with 6TB per month, which is the upper end of my requirements, just to give us a little headroom. To compare USD to EUR, I use today's exchange rate of 0.95 EUR/USD.
The IDrive flex plan is quickly undercut by the Hetzner storage boxes after only 7 months (or 42 TB). However, at the end of the two years, the cheapest (and more managable) option is the IDrive e2 yearly plan. At that point, I would have paid a bit over 4100€ for storage but for the third year alone, I would have to pay that same amount again and so the cost is quickly rising.
So for the long term, would it be cheaper to build my own storage server? I live in a small appartment and don't have the space or a fast enough internet connection to host it at home. Is it feasible to colocate it at a datacenter? Let's try to do the math:
Colocated "Local" Storage
Recurring Colocation Cost
The cheapest colocation option I could find close to my home is at Hetzner in Nuremberg. They offer 14 unit rackspace for 120€ / month + 0.5355€/kWh for electricity and 1.19 € / TB for outgoing traffic over 2TB.
I expect an average power usage of 85 W + 6 W / drive, which would amount to a cost of about 34 + 0.75*NDRIVES € / month for electricity. Outgoing traffic will rarely be more than 2 TB / month, but let's calculate with 5TB / month to be on the safe side, which means ~4 € / month for traffic. Additionally, I would like to have IPv4 connectivity on my server, so I will have to pay an additional 16.18 € for a /29 subnet. Let's also assume 3 trips to the datacenter with my car and 4x 30 min remote hands per year, which would be another 12,50 € + 16,67 € = 29,17 € / month.
Total monthly cost per month for colocation: 207 - 222 €, depending on the number of installed drives.
Server Hardware
I could fit a lot of storage into 14 rack units, but I would need a suitable server case. Many people recommend to get used supermicro or similar server hardware, but those are hard to find and expensive in Germany. Instead, I found a 24 bay hot-swappable 4 unit 19" case on alibaba, which looks exactly like the Inter-Tech 4U-4424 but is much cheaper. The downside of buying on alibaba is that shipping takes a look time (almost two months). The case is compatible with ATX mainboards and power supplies, which allows me to save a lot of money using consumer hardware. I do anyway not care for redundancy for high availability or ECC-RAM. With this case I could build a server that meets (and actually exceeds) all my requirements for less than 1300€, HDDs excluded:
| Item | Price (incl. tax + shipping) |
|---|---|
| Case (alibaba version of Inter-Tech 4U 4424) + sliding rails | 314.17 € |
| Mainboard NZXT N5 Z690 Black | 109.00 € |
| 2x 32GB G.Skill Aegis DDR4-3000 CL16 RAM Kit | 98.56 € |
| 750W NZXT 80+ Gold C Series V2 PA-7G1BB-EU Power Supply | 108.99 € |
| 2x 2TB Samsung 970 EVO Plus M.2 NVMe SSD | 179.82 € |
| Intel i5 12600k CPU | 249.00 € |
| Arctic i35 CPU Cooler | 22.80 € |
| 2x Dell H200 SAS controller | 47.32 € |
| IBM 46M0997 SAS Expander | 33.00 € |
| 4x SSF-8087 to SFF-8643 SAS Cable 0.5m | 44.78 € |
| 2x SSF-8087 to SFF-8643 SAS Cable 0.8m | 22.99 € |
| 2x SSF-8087 to SFF-8087 SAS Cable 0.25m | 21.99 € |
| Total | 1252.42 € |
Hard Drives
With a little patience I was able to find 18 TB enterprise HDDs from private sellers with 3+ years warranty. In particular, I bought 4x 18 TB + 1x 20TB drives for 995.49 € (incl. shipping), which should be enough for the first year. I will use 20TB drives as parity drives (more on that in a later part). If these prices stay constant (they are more likely to drop over the years), I expect to pay about 20x 190 € + 4x 240 € = 4760 € for 24 HDDs, giving me 360 TB usable storage (enough for 5 years) and 4 parity drives.
Other One-Time Cost
There are a few more things to buy: First of all, there is a setup fee for the colocation. Moreover, I want a KVM console to be able to remotely access or powercycle the server in case something goes wrong. Luckily, there is a cheap option called PiKVM, which is based on a Raspberry Pi. I have lots of RPis laying around anyway, so I used one of those for "no cost". I will also need a network switch to connect both, the server and the PiKVM, to the internet. A simple unmanged switch would be enough. There are a lot of cheap used rackmount switches on ebay, but they use a lot of power, so I opted for a popular TP-Link 8 port switch and 3D-printed a nice 19" mount for it. I also ordered spares for the SAS-backplanes, because I don't want to wait for them to ship from china in case one fails. Unfortunately, I forgot to order them together with the server case, so I had to pay again for shipping. I guess 4 spares is a bit excessive but whatever.
| Item | Price (incl. tax + shipping) |
|---|---|
| set-up fee for colocation and /29 ipv4 subnet | 161.53 € |
| Geekworm KVM-A3 Kit for Raspberry Pi 4 | 84.89 € |
| Raspberry Pi 4 | 0.00 € |
| Geekworm power supply 20W 5V 4A | 12.89 € |
| TP-Link TL-SG108E Managed Switch 8 Port Gigabit | 26.60 € |
| Rack Power Delivery Unit | 17.90 € |
| 4x Spare Backplanes | 99.03 € |
| 2x 2m CAT6 Ethernet Cable | 7.98 € |
| M6x16 Cage Nuts 30 Pack | 6.99 € |
| 50x #6-32 UNC x 6mm HDD screws | 5.14 € |
| Zivacate 120 velcro straps | 6.49 € |
| Total | 429.44 € |
Comparison
Now that we have a good estimate for the cost of colocating storage in a datacenter, let's compare it to the cloud options:
If I intend to keep the data for more than 36 months, it is cheaper to colocate my own storage.
For this comparison, I added the cost of the dedicated server to the cloud storage options (47 € / month). If I colocate my own, I can get rid of that server. Moreover, I assumed that I buy a new drive every 3 months. This would save electricity (and HDD-wear) for unused storage capacity, maximize warranty periods and take maximum advantage of decreasing HDD prices (although the calculation assumes constant prices). In reality, it is more practival to buy a couple of drives once a year.
At the end of the 5 year period, when the server is full, I will have saved more than 4000 € - enough to replace a lot of failed hardware parts, if needed. From then on, ignoring hardware fails and assuming no further growth after 5 years, the monthly cost for the colocation would be 222 € (0.62 €/TB) compared to 571 € (1.59€ / TB) with e2. Even if I assume a 20% HDD failure rate after 5 years and assume that I bought all HDDs at once in the beginning and that HDD cost remains the same - all very pessimistic scenarios - the replacements would only increase the cost to 0.84 €/TB, so I think the risk is managable.
It would have been even cheaper to use Hetzner storage boxes or e2 in the first year and then migrate to local storage when the cost exceeds the recurring colocation cost, but I didn't want to bother dealing with two setups and storage migration.
Also in favor of local storage are faster access times and not having to worry about the upoad/download bandwidth to the cloud. Of course it is possible that cloud storage providers could become much cheaper within the next 5 years, but I doubt they will undercut 1€ / TB. I also entirely ignored the value of my own time and labor, but I personally consider this to be a plus, because it is a fun a hobby of mine that I very much enjoy spending time on.
In part II I will explain the hardware selection and the configuration of the server.
Disclaimer: The prices listed in this post are what I paid at the time and may be out of date.