The Lenovo ThinkSystem SD650 is certainly an interesting machine. As a 6U machine with four Intel Xeon Scalable CPUs per rack U it has a similar density to many 2U 4 node chassis. Unlike many of these designs, it is designed for higher wattage parts in higher density rack and aisle configurations by incorporating water cooling into the system.
Lenovo ThinkSystem SD650 Overview
One of the key aspects to the Lenovo Lenovo ThinkSystem SD650 is its six U design. Looking at the front of the chassis we can see features like a custom KVM connector, management and data LAN ports, USB ports and an expansion slot for Omni-Path and Mellanox FDR (and future) Infiniband.
Each U of the 6U chassis has on tray. Each tray holds 2 complete dual CPU systems for a total of four CPUs per U as well as some provision for expansion slots and storage.
Perhaps the first thing readers will notice is that there are no fans in the chassis which essentially removes all of the moving parts from a system, increasing reliability. Cooling instead is provided via a water cooling network that snakes through the two systems.
Lenovo provided a diagram showing how the cold water (blue) flows through the CPUs and memory as well as cooling the fabric cards and storage. RAM modules tend to use relatively little power, however NVRAM and Optane DIMMs use more power. While these are easy to cool with air cooling, if one removes fans from the mix, a cooling solution is required.
Using water cooling allows Lenovo to increase reliability as well as more efficiently cool servers. Using water cooling and heat exchangers can have massive cooling power savings, especially using free cooling heat exchangers rather than traditional chillers. More data centers are being built with water cooling facilities and HPC installations are certainly targeting water cooling for next-generation builds.
Storage is relatively minor. One can fit two 2.5″ SATA 7mm SSDs or a 2.5″ NVMe SSD in a standard 15mm form factor. One can also use m.2 SATA SSDs. In these systems, this storage is for tasks like holding OS images and some local data.
The rear of the chassis has six 1.3kW power supplies that also hold the systems fans. Not everything is water cooled. The rear also has the fan controllers and the water cooling manifolds that connect to facility water distribution sources.
Another key to this design is that it is intended for “future” processors with 245W TDP, not just the 205W TDP Intel Xeon Platinum 8180 that we have today. As TDP raises, and higher power devices invade RAM slots facilities have to deal with more heat in aisles.
Using free cooling and water-based heat exchangers is seen as a key technology to remove heat from higher power racks. Lenovo says that the 72 system rack with six of these 6U systems is expected to consume 43kW per rack.
Lenovo ThinkSystem SD650 Key Specs
Here are the key specs for the system:
Full-wide 1U tray (six per n1200 Enclosure)
NeXtScale n1200 Enclosure (6U)
2x Intel® Xeon® Scalable Processors per node; 2x nodes per 1U tray
Up to 768GB using 12x 2667MHz TruDDR4 DIMMs per node
1x 50mm width ML2 slot and 1x PCIe x16 slot for EDR InfiniBand or Intel Omni Path, per server node
|Internal Storage||Up to 2x 2.5” SATA SSDs (7mm height) or 1x 2.5” NVMe SSDs (15mm height) per node; optional M.2 SSD Hardware RAID adapter, supporting M.2 drives|
|RAID Support||Onboard SATA controller with Software RAID; optional Dual M.2 SSD adapter with Hardware RAID 1|
|Network Interface||1x 1GbE BaseT NIC per node; additional high-speed network adapters (InfiniBand or Omni Path) can be installed in available front-accessible PCIe x16 adapter slot|
|Power Management||Rack-level power capping and management via Extreme Cloud Administration Toolkit (xCAT)|
|Systems Management||Remote management using Lenovo XClarity Controller; 1Gb dedicated management NIC plus 1Gb shared management NIC per node|
|OS Support||Red Hat, SUSE, CentOS (with LeSI support); Visit lenovopress.com/osig for more info.|
|Limited Warranty||3-year customer replaceable unit and onsite limited warranty, next business day 9×5, service upgrades available|