This is perhaps one of the biggest pieces of hardware. NVIDIA has a BlueField-3 DPU paired with an NVIDIA A100 GPU in a massive card. One way to look at the card is that it is “simply” an accelerator card. The other way to look at the NVIDIA AX800 is it is a server with 16x Arm cores, 32GB of memory and 40GB of storage, an ASPEED BMC, an A100 GPU, and over 400Gbps of total networking, plus NVLink on a single card.
NVIDIA AX800 High-End Arm Server on a PCIe Card
The new NVIDIA AX800 combines two main components:
- A NVIDIA BlueField-3 DPU
- A NVIDIA A100 80GB
Both of these are combined into a single dual-slot card with a 350W TDP and a minimum of only 150W TDP. This is up from the 300W of the previous generation NVIDIA A100X, but while the GPU remains the same, we get a much faster DPU.
The fun part about this card is that the I/O panel looks like a NIC with the two QSFP56 ports and the out-of-band management port for a 1GbE connection to the ASPEED AST2600 BMC. We will quickly note here that NVIDIA is using two QSFP56 cages each at 200Gbps VPI so they can run in Ethernet or InfiniBand mode. We recently did our NVIDIA ConnectX-7 400GbE and NDR Infiniband Adapter Review and that utilized OSFP cages which were less than exciting to work with.
As much as the rear looks like a NIC or a DPU, the rest of the card looks like a professional NVIDIA GPU even with features like NVLink on top. The NVLink on this card needs to be linked directly to another card and is not meant to link to multiple GPUs simultaneously.
Still, the card has some impressive specs. From what we can understand, this is not an integrated DPU-GPU at the die level since we can see two packages on the board. Instead, this is a PCIe-connected card that combines the specs of a GPU and DPU onto a single card. Here are some of the highlights:
Key NVIDIA A100 GPU Specs
- GPU generation: NVIDIA A100 “Ampere”
- GPU base clock: 840MHz
- GPU maximum boost clock: 1.41GHz
- GPU memory clock: 1.593GHz
- GPU memory type: HBM2e
- GPU memory size 80GB
- GPU memory bus width 5,120-bits
- GPU peak memory bandwidth Up to 2,039GB/s
Key NVIDIA BlueField-3 DPU Specs
- DPU generation: NVIDIA BlueField-3
- DPU core generation: Arm v8.2 A78
- DPU core quantity: 16
- DPU memory clock: 5,600MHz
- DPU memory type: DDR5
- DPU memory size: 32GB
- DPU memory bus width: 128-bits
- DPU peak memory bandwidth: Up to 89.6GB/s
- DPU storage: 40GB eMMC
- DPU accelerators:
- Hardware root of trust
- IPsec/TLS data-in-motion encryption
- AES-XTS 256/512-bit data-at-rest encryption
- SHA 256-bit hardware acceleration
- Hardware public key accelerator
- True random number generator
- DPU BMC: ASPEED AST2600
To recap, this card has:
- A 16-core Arm CPU, with its own 32GB of memory, and a 40GB eMMC boot drive
- Dual 200Gbps Ethernet/ InfiniBand networking
- A NVIDIA A100 80GB GPU
- NVLink for a high-speed interconnect
- A BMC for out-of-band management
- A PCIe connector
In most cases, we would consider that a complete server and a fairly high-end one at that. This is one of the most unique DPUs or GPUs out there that really does not fit neatly into a segment. NVIDIA is targeting these at the Telco markets, but it is one of those products that we wish had broader availability because it feels like something that could have a PCIe switch chip hung off of it to make amazing AI-enabled edge server nodes.
As a fun aside, we are covering it in June 2023, but we have had this draft in progress since Computex 2023 in May. It just, somehow, was released and got no coverage. Hopefully, this helps the card get a bit more exposure.