STH Hardware Upgrades 2025 Edition

I wanted to introduce a little series that we will be running over the next few weeks. The goal is to discuss this year’s hardware refresh cycle at STH on several fronts. We have several projects that are either in progress or have recently been completed. Some of them did not go as we originally planned.

STH Hardware Upgrades 2025 Edition

We have four (maybe five) good topics that I think folks will be interested in.

Hard Drives No More?

We still have a handful of hard drives in our production clusters, and a few more for storage in the studio. You may have seen our Farewell Brave 8TB WD Hard Drive You Served STH Well piece in 2024.

We have a project going on to replace all of those hard drives with SSDs. Even with 50-100 hard drives, you end up having failures every year, and since the technology exists to just get SSDs, it is a project we have been focusing on.

Right now we are paying something like 3-4x to move to all-flash. There is probably wisdom in not doing the project, but I just want the reliability.

The Ampere Altra (Max) Project

I think folks know that I have a strong belief that if we test something and give the OK, we should be willing to use it. Earlier this year, as part of building out the NVMe storage arrays, we went and bought a few Ampere Altra (Max) platforms.

The platforms performed as we would have expected, but then the unexpected happened and this project took some turns I was not expecting. I think we learned a few things here that are probably applicable to a lot of our readers that are worth getting into.

Ending Google Drive Sprawl

I am not entirely sure why, but at STH, we have been using Google Drive for a long time, but that practice has come under fire.

We have our own hosted storage for the lab, another storage setup for backups, and individual setups for in-lab storage and our studio production storage. All of those have different optimizations, from fast lab storage to slower and lower-cost backup storage to quiet studio storage. However, to transfer photos and footage from one person to another, we have primarily been using Google Drive. During the pandemic, we needed a video sharing platform and Google Drive was there, and easy, so it started being used. Frankly, I think there are a lot of tools in other businesses that similarly find their way into their respective stacks.

Our challenge now is that footage needs are expanding. Even if we send compressed footage (like 4K h.265) a small project might be 200GB. A larger project can easily exceed 1TB. This summer we are bringing onboard a second editor, and then a part time shorts editor, to try getting the production stabilized, and so the number of in-flight projects is going up. We are also working on another YouTube set that Sam and I have been prototyping layouts for in the studio in an approximately 250 sq ft area. The goal will be having some new capabilities including new camera angles. The downside, is that moving to that setup will create even more storage demands in our production.

It is time to do bust some of these storage silos and pull the easy mode Google Drive out of its current role. Just because it was easy to setup and maintain does not mean it is the correct answer as we will start needing much more storage in the future. Also, the 5TB to 10TB jump is actually $249 to almost $600 since we need to get off an annual billing plan. It may not seem like much until the next time prices increase.

Fixing the Load Generation Problem

At STH, our networking capabilities need to be enhanced. I know this. Rohit knows this. I think everyone has had that realization. The challenge is generating high-end traffic across all the products we test today and will continue to test through at least 2026. That means we need to scale from 1GbE devices to 800GbE. At the same time, I have been sick of iperf3 for some time.

Back in 2021, we had a small series with devices like the Ubiquiti EdgeRouter ER-12P that used a new setup. We bought a Dell Precision 7920, outfitted it with higher frequency CPUs and NICs down to the point we had NICs and everything setup in Cisco’s Trex traffic generation tool making cool charts like this one:

The big challenge was that the setup did not scale super well and Trex is more of a bear to maintain. We even went sofar as to get the Napatech FPGA solution working but with Trex, it is very rough, and one of our fancy and specific transcievers failed.

The new load generation machine will be based on the Supermicro Hyper SuperServer SYS-222HA-TN that we reviewed, but with a very fancy configuration both on the hardware and software side. This is a huge (HUGE?) investment, but I think STH readers are really going to like what we have on the network testing side later this year.

The Stretch Goal

One other upgrade we might do in 2025 is do some colocation upgrades beyond what is mentioned above. We actually have machines that have been installed in some locations that have not been transitioned to production use yet. In a 2025 dominated by AI headlines, maybe this is the least exciting thing to see.

Final Words

Years ago, on STH, we used to discuss the hardware upgrades we would make more frequently. Now, we are deploying things so quickly, and the labs are churning so fast, that we got a bit behind and need to catch up on some of the lower-priority items. Since we have been mixing in “home run” projects that went splendidly with projects that went less well because of a mid-deployment re-think, I think we have some really neat key lessons learned that would make fun articles.