In the world of storage, perhaps nothing says built for a specific purpose like the top-loading storage server segment. When I heard that Supermicro had a new and updated design, and I knew we would be doing a day at Supermicro looking at products, I thought this is a platform we needed to look at. Normally, we would cover the platform at trade shows, but without trade shows in 2020, that is hard to get hands-on time. Instead, I drove over to Supermicro headquarters and checked out the new system. That allowed me to get some hands-on time without having to move a server that weighs more than a person around a data center.
This is part of our visit to the Supermicro HQ post-lockdown series. We previously looked at the Supermicro SYS-240P-TNRT 4P 3rd Gen Xeon Scalable Server which we showcased alongside the Cooper Lake launch. We also looked at the Simply Double platform. As a part of that series, we have an accompanying video:
We are going to have more detail in this article, but want to provide the option to listen. As a quick note, Supermicro filmed the video bits at their HQ, provided the systems in their demo room, and their product managers that were able to make it. We did a whole series but are tagging this as sponsored since we relied upon their facilities instead of our own. I was able to pick the products we would look at and have editorial control of the pieces (nobody is reviewing these pieces outside of STH before they go live either.) In full transparency, this was the only way to get something like this done, including looking at a number of products in one shot, without going to a trade show during shelter-in-place. Look for more in this series coming to STH over the coming weeks.
New Supermicro 60-Bay Top Loader
The new Supermicro 60-bay top-loading system we are looking at has a model number of SSG-6049SP-DE1CR60 which is very long and perhaps fitting for such a massive machine. We are just going to quickly note that Supermicro has other configurations including a 90-bay server and JBOD versions of these systems. To be frank, the 60-bay version was easier to move but much of the same principles apply to the other products as well.
Perhaps the highlight feature of the new top-loading design are the 60x 3.5″ drive bays. Typical front-mounted 4U storage servers get 24x 3.5″ bays. One can find 4U platforms with both hot and cold aisle storage that can utilize 36x 3.5″ individually serviceable bays. For higher densities, top-loading designs are very popular. In Supermicro’s new design, there is a new drawer design. The main chassis stays in the rack but a drawer with four rows of fifteen drives each can slide out to be serviced.
In this revision, Supermicro made a big push for serviceability. Even the SAS expanders are hot-swappable. Here is a quick photo of one in the disk drawer partially removed next to one that is still in the system.
As part of the company’s green IT push, it is using a design where the main chassis can stay installed but other parts swapped and upgraded. We were looking at the compute module which can be removed from the chassis completely.
The compute module has features such as room for boot drives, caching drives, PCIe add-on cards, and a full dual Intel Xeon Scalable server. One can use Optane DCPMM (or Optane PMem 100) modules in the server as well as this is an eight DIMM per socket design.
There are even a pair of gold SATA ports for those who want to boot off of SATA DOMs that are powered from the motherboard. Taking a look at the connector side of the I/O module, one can see that this is on a completely new level of design than Supermicro’s original 60-bay servers. There are multiple levels of PCB that help fit all of the I/O needed to run the chassis. The M.2 SSDs, for example, sit on a PCB that is connected via a vertical high-density connector to another PCB between the drives and the dual Xeon Scalable processors.
By leveraging these higher-density and hot-pluggable connectors, the new 6-bay server can be removed from the system, and serviced easily without disturbing the spinning drives. These high-density connectors lead to the internal cable management arms that feed power and data pathways to the storage drawer.
The system we saw was a single controller configuration. In that configuration, the main controller node sits in the bottom removable section. Above it, there are fans as well as six 2.5″ bays.
With the new 60-bay top-loading design, Supermicro has a module that sits atop the compute module providing fans, but also 2.5″ storage. One can add, for example, two 2.5″ SATA boot drives to make those hot-swappable (the internal M.2 drive are not) as well as four 2.5″ U.2 NVMe SSDs for caching or a faster storage tier.
That brings us to what may be the most intriguing addition to the new design. The new solution can handle two nodes in a single chassis by replacing that I/O and fan module with a second dual Intel Xeon Scalable controller node. One way this can be used is simply as a second server to add more compute, network, and memory to the 4U footprint. Supermicro has a link between the two controllers (SBB) which can allow high-speed synchronization for active-active configurations.
With this design, the new Supermicro 60-bay top-loading storage servers, along with their 90-bay and JBOD counterparts, get an enormous amount of flexibility. One can have JBOD chassis, single controller nodes, and even dual controller nodes for active-active or active-passive setups.
These are not small systems. One of the challenges with top-loading storage designs is that the chassis tend to be very large which requires proper depth and even proper strength racks and data center floors. The 60-bay unit we looked at is not the densest 4U system on the market. Supermicro even has a 90-bay variant as an example. Instead, it is a useful compromise for a storage solution that will easily integrate into more existing data center infrastructures while still providing a fairly massive bump in density.
Let us be clear, these new 60-bay storage servers are an enormous advancement in both capabilities, but also serviceability compared to the original iterations we saw years ago. It is great to see the segment mature with new building blocks for scale-up and scale-out storage. One of the challenges of top-loading storage servers has always been serviceability. With the new design, the server chassis sits in the rack and can be upgraded without having to remove it from the rack. That is much more like a blade server compute equivalent.