In the world of storage, perhaps nothing says built for a specific purpose like the top-loading storage server segment. When I heard that Supermicro had a new and updated design, and I knew we would be doing a day at Supermicro looking at products, I thought this is a platform we needed to look at. Normally, we would cover the platform at trade shows, but without trade shows in 2020, that is hard to get hands-on time. Instead, I drove over to Supermicro headquarters and checked out the new system. That allowed me to get some hands-on time without having to move a server that weighs more than a person around a data center.
Video Version
This is part of our visit to the Supermicro HQ post-lockdown series. We previously looked at the Supermicro SYS-240P-TNRT 4P 3rd Gen Xeon Scalable Server which we showcased alongside the Cooper Lake launch. We also looked at the Simply Double platform. As a part of that series, we have an accompanying video:
We are going to have more detail in this article, but want to provide the option to listen. As a quick note, Supermicro filmed the video bits at their HQ, provided the systems in their demo room, and their product managers that were able to make it. We did a whole series but are tagging this as sponsored since we relied upon their facilities instead of our own. I was able to pick the products we would look at and have editorial control of the pieces (nobody is reviewing these pieces outside of STH before they go live either.) In full transparency, this was the only way to get something like this done, including looking at a number of products in one shot, without going to a trade show during shelter-in-place. Look for more in this series coming to STH over the coming weeks.
New Supermicro 60-Bay Top Loader
The new Supermicro 60-bay top-loading system we are looking at has a model number of SSG-6049SP-DE1CR60 which is very long and perhaps fitting for such a massive machine. We are just going to quickly note that Supermicro has other configurations including a 90-bay server and JBOD versions of these systems. To be frank, the 60-bay version was easier to move but much of the same principles apply to the other products as well.
Perhaps the highlight feature of the new top-loading design are the 60x 3.5″ drive bays. Typical front-mounted 4U storage servers get 24x 3.5″ bays. One can find 4U platforms with both hot and cold aisle storage that can utilize 36x 3.5″ individually serviceable bays. For higher densities, top-loading designs are very popular. In Supermicro’s new design, there is a new drawer design. The main chassis stays in the rack but a drawer with four rows of fifteen drives each can slide out to be serviced.
In this revision, Supermicro made a big push for serviceability. Even the SAS expanders are hot-swappable. Here is a quick photo of one in the disk drawer partially removed next to one that is still in the system.
As part of the company’s green IT push, it is using a design where the main chassis can stay installed but other parts swapped and upgraded. We were looking at the compute module which can be removed from the chassis completely.
The compute module has features such as room for boot drives, caching drives, PCIe add-on cards, and a full dual Intel Xeon Scalable server. One can use Optane DCPMM (or Optane PMem 100) modules in the server as well as this is an eight DIMM per socket design.
There are even a pair of gold SATA ports for those who want to boot off of SATA DOMs that are powered from the motherboard. Taking a look at the connector side of the I/O module, one can see that this is on a completely new level of design than Supermicro’s original 60-bay servers. There are multiple levels of PCB that help fit all of the I/O needed to run the chassis. The M.2 SSDs, for example, sit on a PCB that is connected via a vertical high-density connector to another PCB between the drives and the dual Xeon Scalable processors.
By leveraging these higher-density and hot-pluggable connectors, the new 6-bay server can be removed from the system, and serviced easily without disturbing the spinning drives. These high-density connectors lead to the internal cable management arms that feed power and data pathways to the storage drawer.
The system we saw was a single controller configuration. In that configuration, the main controller node sits in the bottom removable section. Above it, there are fans as well as six 2.5″ bays.
With the new 60-bay top-loading design, Supermicro has a module that sits atop the compute module providing fans, but also 2.5″ storage. One can add, for example, two 2.5″ SATA boot drives to make those hot-swappable (the internal M.2 drive are not) as well as four 2.5″ U.2 NVMe SSDs for caching or a faster storage tier.
That brings us to what may be the most intriguing addition to the new design. The new solution can handle two nodes in a single chassis by replacing that I/O and fan module with a second dual Intel Xeon Scalable controller node. One way this can be used is simply as a second server to add more compute, network, and memory to the 4U footprint. Supermicro has a link between the two controllers (SBB) which can allow high-speed synchronization for active-active configurations.
With this design, the new Supermicro 60-bay top-loading storage servers, along with their 90-bay and JBOD counterparts, get an enormous amount of flexibility. One can have JBOD chassis, single controller nodes, and even dual controller nodes for active-active or active-passive setups.
Final Words
These are not small systems. One of the challenges with top-loading storage designs is that the chassis tend to be very large which requires proper depth and even proper strength racks and data center floors. The 60-bay unit we looked at is not the densest 4U system on the market. Supermicro even has a 90-bay variant as an example. Instead, it is a useful compromise for a storage solution that will easily integrate into more existing data center infrastructures while still providing a fairly massive bump in density.
Let us be clear, these new 60-bay storage servers are an enormous advancement in both capabilities, but also serviceability compared to the original iterations we saw years ago. It is great to see the segment mature with new building blocks for scale-up and scale-out storage. One of the challenges of top-loading storage servers has always been serviceability. With the new design, the server chassis sits in the rack and can be upgraded without having to remove it from the rack. That is much more like a blade server compute equivalent.
Background music in video is there on purpose? Very distracting to me…
DaleQ great feedback. We got comments last time that people wanted music. We will tone it down in the next video.
Thanks Patrick for your reply. For me, I can’t understand why I’d want to add noise/distraction (music) to your excellent content. Keep up the great work! Glad I can still read articles.
Very cool
This thing is friggen sweet! I am very curious to see where these units wind up in the market. This would make an absolutely killer SAN. Any word on OS support?
I would hate to have to put one of these things in a rack though. Talk about team lift. lol.
Supermicro clearly trying to get in on 45Drive’s game!!!
Having had hands-on with both, the SM design is in a different league.
Being able to pair this with single socket AMD Epyc nodes would be awesome (e.g. for Ceph).
Just wanted to say I had commented a while ago about your videos asking you to speak up a bit (and sound confident, damnit!) And I just thought I’d say that video quality has gone up greatly since then….sure incremental improvements like music and volumes are great but yea just overall the videos are just fantastic especially compared to those first two or three….plus keep making the excellent content both here and on the tube!!!
You commented upon the weight of the unit but not the actual weight with 60 drives. How much should we plan for?
@Dave, a Nexenta unit (which is basically a supermicro chassis) with 90 drives fully loaded weighs something like 280Lbs, so figure somewhere around there, maybe 240-250 with 45 drives?
Hey SM: Now make us a 2.5″ version for an all-flash server
WRT lifting: As Patrick says – server lifters are necessary for this work
There are 2 primary makers in this market – Serverlift and Tawi
Serverlift are reasonably priced in the USA (a SL500 will cost you less than a single one of these nodes) but the overseas distributors GOUGE like you wouldn’t believe – resulting in a $11k lifter (USA) being marketed at nearly $20k in europe and $40k in australiasia – and ServerLift was so focussed on the USA market they thought these prices were reasonable outside North America (small wonder that their EMEA agent sold _1_ unit into the UK across 2000-2005 when the market in London alone is 100 lifters)
Tawi’s units are repurposed lifting devices from the food handling market and have a lot of downsides for operating in space-constrained server rooms – they’re also hideously expensive.
A STH article on lifters would be a good move 🙂 (Hint Hint!)
With regards to overseas (relative to the USA) pricing, perhaps those complaining about it should look at the tax structures imposed by those non-USA countries.
Some countries outside of the USA are taking up oppressive tax rates against some or all types of imports from the USA.
At the prices suggested by Alan Brown you would think some non-USA inventors would create their own form of this product. I say “create” because of the legal implications that exist in many (not all, especially not in certain Asian locales) countries for copying an existing product.
Patrick: About the background music, the problem is that the level is constant and too low; so there’s this jingling in the background that we can’t understand, like a second conversation or noise.
The ‘secret’ to having a music track with an announcement track is to use ducking. Ducking lowers the volume of the music just before you speak to a background level, letting your voice come through clearly, then after you finish speaking the music comes back to a comfortable level.
Of course you need an interesting script and ‘elevator music’ with a wide appeal. The music should be free of vocals and “easy listening”, unless you are talking about Monster Trucks, etc.
Here’s a couple of videos describing ducking using the Digital Audio Workstation (DAW) called Reaper (it’s free). Here are different ways to do this, varying in the feel or ambience that each offers:
https://youtu.be/8Tt5yWOxB2E
https://youtu.be/8xcPpKe_sgY
https://youtu.be/pGZZGP35JVM
And, a tutorial about Compression (volume leveling): https://youtu.be/yi0J9JsRdI4
That should give you a professional DIY sound.