A Tour of the QCT Intel Sapphire Rapids System Overload Booth

At Intel Innovation 2022, QCT brought its new 4th Generation Intel Xeon Scalable servers (codename Sapphire Rapids.) Some vendors at the show brought a machine or two. Others just showed videos of products. QCT on the other hand had a huge array of next-generation systems, and so I wanted to show some of the cool designs in the booth.

A Tour of the QCT Intel Sapphire Rapids System Overload Booth

The first eye-catching solution was the 42U QCT Liquid Cooling rack solution. On the show floor, the company was running the QCT D54Q-2U in liquid and air cooled configurations.

The servers are liquid-cooled via direct to chip coldplates, then the rear door heat exchanger is used to more efficiently remove heat from the systems. I ended up taking 100+ photos of just this unit so we are going to do a deeper dive in the next week or so. We have been big on liquid cooling for the last year plus on STH, and so this was an awesome demo.

QCT had the liquid cooled server open in its booth as well. One can see the blocks here on each CPU. There are then hot and cold tubes that go to the rear of the chassis and then connected to the rest of the rack.

In these servers, QCT is still using air to cool the memory, storage, and networking. It seems like high-power accelerators can be added to liquid cooling loops if needed. The net impact is that the fans spin at a much lower RPM and therefore they use a lot less power than if they were at full speed.

Here is the air shroud in the QCT QuantaGrid D54Q-2U air cooled.

This was just interesting to see an air v. liquid cooled solution out on the show floor, especially with next-generation platforms.

For the 1U market, QCT had a dense offering. The QCT QuantaGrid D54X-1U is the 1U server offering with sixteen DIMMs per CPU.

A cool aspect of the company’s new motherboards is how the rear I/O is laid out. QCT has dual OCP NIC 3.0 slots, one on either side. In the middle, there is another module that has the BMC and rear I/O.

Above the OCP NIC and rear I/O, there are two full-height risers. We were told that there are other configurations available as well.

What is clear, however, is that the system is packed. If you are running Skylake or Cascade Lake servers, this generation will be a >2:1 consolidation ratio opportunity over those servers.

Along the theme of 1U servers, we have the QCT QuantaGrid D54T-1U. This is a platform that many of our readers have seen previous versions of, and many have also used. This time, however, there is a big change.

The front of the system has a pull-out tray (more on this shortly) and the cool QCT feature of having space for four cache SSDs below.

On the rear of the system, we can see expansion card slots, but also room for things like OCP NIC 3.0 cards.

Looking at the tray, we can see twelve 3.5″ hard drive spaces. Each tray has a cool latching mechanism and is similar to previous generations of this chassis.

What is different, is the CPU. With the Sapphire Rapids Xeon, this platform can utilize a single CPU rather than requiring two CPUs just for the extra PCIe lanes. That is a huge cost savings for a server in this class.

Next to the 1U storage server, there was ain interesting looking 3U system, the QCT QuantaGrid D54U-3U. This is a GPU and accelerator platform. Here we can see the 10x 2.5″ hot swap bays and then front hot-swap fans.

The rear has room for up to four PSUs, an OCP NIC 3.0, and the rear I/O. There is more however.

Each side of the chassis has two risers. These risers use quick mating connectors to pop in-and-out. There are even small cutouts in the chassis and the riser cages to support bridges, for example, NVLink bridges for GPUs.

The middle of this system is for networking and other I/O. QCT told us that the 3U design in this generation was to provide more efficient cooling. With 4x 350W PCIe GPUs, 2x higher power Xeon Scalable CPUs, plus all of the other components in the system, the decision to use 3U was for efficiency. Also, even using these 3U servers, one can easily top 30kW of compute in a rack so QCT expanded the chassis size from 2U to 3U for better effiicency.

For higher-end GPU and AI accelerators, there was a massive 7U compte box. where 5U was dedicated solely to cooling.

Here is the rear of the unit. If you see there is a top 4U section, then three distinct 1U portions. That top 4U section is designed for things like the HGX H100 baseboard. One can think of this like QCT’s version of the NVIDIA DGX H100. We are not showing that baseboard because inside was a prototype OAM platform that one would use for accelerators like Ponte Vecchio or the Gaudi 2. QCT’s design can accomodate different types of accelerators, although the H100 seems like it will be the leading configuration.

Below the GPU/ accelerator tray is the motherboard. Here we get two Sapphire Rapids socket each with 16 DIMMs. This motherboard connects to the PCIe switch midplane and is removable for service.

Below this on the rear of the system one can see 1U of blanks. The first is an array of up to 10x OCP NIC 3.0 slots. The bottom 1U in this system is for DC power supplies.

Final Words

At Intel Innovation 2022, vendors brought a lot of next-generation Intel Xeon platforms. QCT, however, was a standout with just how many different types of systems that the company had on display. I thought thi swould be a good opportunity to show some of the new systems including standard 1U/2U servers, liquid cooled servers, a 1U storage server, and GPU servers. If you want to see more on the liquid cooled rack solution, we should have that one out in about a week.

The 4th generation Intel Xeon Scalable CPUs are still a ways away. At least Intel let vendors show off some platforms and us to share them at Innovation 2022. We will have an accompanying video soon with some other quick views of these and other servers.