Modern AI servers are remarkable feats of engineering. Each successive generation raises the bar not just on raw GPU performance, but on the complexity of the systems required to support those GPUs. Networking, power delivery, liquid cooling, and software management have all evolved dramatically, making it genuinely challenging to keep track of all the changes. To address that, we visited Supermicro at its headquarters in San Jose, California, for a comprehensive tour of the major systems and infrastructure components that make today’s AI factories possible.
The scope of Supermicro’s in-house design and manufacturing is striking. The company is not simply integrating third-party components into a chassis. From the server nodes themselves to the cold plates, cooling manifolds, in-row CDUs, rear door heat exchangers, power shelves, and even large outdoor cooling towers, Supermicro is engineering and producing the full stack. Supermicro is doing this to deliver NVIDIA B300-generation solutions at scale. I asked Supermicro and NVIDIA if we could look at the differences between the NVIDIA B200 and B300 generations with the other Data Center Building Block Solutions that Supermicro makes, supporting its AI Factory efforts.
Of course, we do not have all of this hardware sitting around, and we are filming inside Supermicro’s factory in San Jose, California. We have to say this is sponsored. My thought was that it is worth just showing some of the key changes side-by-side, and so we needed help to get all of the components together. We started with more modest goals, then we just ended up looking at almost everything in the video because we were on site, and Supermicro gave us access to its San Jose, California, factory.
The Two Generations: NVIDIA HGX B300 vs. HGX B200 Air-Cooled Servers
Our first stop covered Supermicro’s air-cooled HGX server line, where the generational comparison between NVIDIA B200 and B300 becomes immediately tangible. Supermicro has offered air-cooled HGX 8-GPU servers for multiple generations, and placing a B200-generation system next to a B300-generation system makes the key changes obvious.
On the B200-generation system, a row of eight discrete NVIDIA ConnectX-7 network interface cards is visible along the bottom of the front panel. Each of those cards operates at 400 gigabits per second and is paired with a single GPU, for a total of eight NICs across eight GPUs. The space above those NICs is dedicated solely to air-cooling hardware.
The B300-generation server looks different immediately. Those eight discrete NICs are gone from the front. That is because the NVIDIA HGX B300 8-GPU baseboard integrates NVIDIA ConnectX-8 networking directly onto the board itself.
Each ConnectX-8 interface provides 800 gigabits per second, and because the NICs are no longer occupying front-panel slots, those 800Gbps ports are instead exposed as direct external ports across the top of the system.
The result is a cleaner front-panel layout and, more importantly, double the network throughput per GPU compared to the B200 generation.
Liquid-Cooled HGX B300 and B200: What Changes and What Stays the Same
Moving to the liquid-cooled variants of these servers, the generational story is similar but told differently. At first glance, a liquid-cooled NVIDIA HGX B200 server and a liquid-cooled NVIDIA HGX B300 server appear nearly identical. A distinction that matters again, beyond the GPUs being used, is the networking.
On the liquid-cooled Supermicro NVIDIA HGX B200 system, slots for discrete NVIDIA ConnectX-7 NICs are visible at the front of the chassis. Eight GPUs, eight NICs, 400Gbps per NIC. That is a proven, well-understood configuration that has served many production deployments well. Note, in the photo below, we did not have the eight ConnectX-7 NICs installed, but they would go in that bottom center area.
On the liquid-cooled Supermicro NVIDIA HGX B300 system, those front-panel NICs are gone, replaced by the integrated NVIDIA ConnectX-8 architecture. One ConnectX-8 per GPU, 800Gbps per GPU, for a total of 6.4Tbps of network throughput across the full 8-GPU system. This is worth emphasizing because most coverage of the B300 generation focuses on the memory capacity and compute improvements (e.g. from 192GB to 288GB of HBM3E.) The doubling of network bandwidth is at least as significant for distributed training and large-scale inference deployments.
Supermicro designs and manufactures the liquid cooling components for these systems in-house. The warm and cool coolant hoses exiting the rear of each server connect to rack manifolds that Supermicro also produces. The cold plates that make direct contact with GPUs, CPUs, and other thermal loads are also Supermicro designs. This level of vertical integration in the cooling stack is one of the key factors that enable faster delivery of complete, validated systems.
The ORV3 B300 NVL8 System: Up to 144 GPUs in a 48U Rack
One of the more distinctive systems on the tour was an Open Rack V3 (ORV3) chassis hosting an NVIDIA B300 NVL8 baseboard. This is a two-unit system with a clear separation of function: the lower unit contains the host node with standard CPUs and memory, while the upper unit houses the NVL8 GPU baseboard itself.
The NVL8 baseboard connects eight B300 GPUs via NVLink, giving those GPUs a shared high-bandwidth interconnect within the node. Each GPU carries 288 GB of HBM3E memory, and the integrated ConnectX-8 networking delivers over 800Gbps of east-west bandwidth out of the node. All of this runs in a liquid-cooled ORV3 enclosure with blind-mate connectors at the rear for both the cooling loop and power delivery.
Because the 2U ORV3 NVL8 chassis is considerably denser than a standard rackmount form factor, a 48U ORV3 rack can accommodate up to 144 B300 GPUs. The shared ORV3 infrastructure with the NVL72 means that investments in power and cooling infrastructure are compatible across both product lines, which simplifies planning for operators who need to deploy both form factors.
PCIe GPU Servers and the Move to ConnectX-8 Switch Architecture
Not every AI workload calls for an HGX system. For mixed workloads that combine inference with tasks such as graphics rendering, VDI, engineering simulation, or other GPU-accelerated applications, PCIe GPU servers remain highly relevant. Supermicro is bringing the same ConnectX-8 networking improvements to this server class.
The new-generation PCIe GPU server uses a ConnectX-8 PCIe switchboard design. Four ConnectX-8 NICs are installed, with each NIC serving two GPUs. Each NIC provides 400 Gbps, giving a total of 3.2Tbps across all four NICs for the full 8-GPU system. That is double the throughput of the older design that used four ConnectX-7 NICs at 400Gbps each, producing only 1.6Tbps total.
The ConnectX-8 PCIe switch architecture also introduces PCIe Gen 6 support, which matters for high-throughput GPUs like the NVIDIA RTX Pro 6000 Blackwell Server Edition. Those GPUs fully utilize PCIe Gen 5 x16 bandwidth, and the ConnectX-8 switch gives them a network interface commensurate with the PCIe slot throughput itself. Supermicro also continues to offer standard 2U rackmount servers with its own riser and motherboard designs engineered to properly power, cool, and connect multiple GPUs in conventional rack infrastructure.
Next, let us get to the Supermicro NVIDIA GB300 NVL72 rack.
I’m in awe of how much you cover. Small networking to 1.6T optical DSP’s. Small GB10 box to giant Supermicro AI Factory.