Broadcom is now on stage at Hot Chips 2026 with the Thor Ultra Ethernet NIC, an 800GbE adapter aimed squarely at AI and HPC fabrics. This Thor Ultra family launched earlier this year, and this session goes deeper on the design behind it.
This one we are doing live, so please excuse any typos.
Broadcom Thor Ultra Ethernet NIC at Hot Chips 2026
Broadcom is setting the stage for Thor Ultra. Ethernet here spans scale-up in the rack, scale-out across racks, and scale-across between data centers, and each layer drives a shared set of NIC requirements. Each needs high bandwidth, enhanced RDMA, low latency, and QoS, joined by virtualization, security, timing, telemetry, and diagnostics. It is a little bit awkward given what NVIDIA’s framewrok will look like later in this session block.
Thor Ultra pairs a PCIe Gen6 x16 host interface with an 8-port 100G MAC, 256 SR-IOV virtual functions, and 8x 100G Serdes, and it offloads RoCEv2, stateless packet processing, and inline encryption. Enhanced RoCE here exceeds 64K queue pairs, with a TruFlow engine and BroadSAFE security.
Broadcom lays out the generational leap from the prior Thor here. What stands out is packet spraying with out-of-order placement, reliability beyond Go-Back-N, simpler, programmable congestion control, and double the ports with multiplane connectivity.
This is the silicon itself, on 5nm, with a 2.4 billion-transistor count in a 27×27 package, at 40-42W max power. TX and RX buffers, an embedded CPU subsystem, and the P4-like programmable engines that carry much of the eRoCE work are all visible in the die. How cool is this! A die shot from a modern NIC!
Broadcom ships Thor Ultra in OCP NIC 3.0 and PCIe form factors configured for one OSFP112 or two QSFP112 ports. An OSFP112 port supports 800G, 2x 400G, 4x 200G, or 8x 100G, so operators can reuse a single board across multiple port densities. Cabling covers DAC, LPO, AEC, and optics, and the board tops out at 50-55W without optics.
Here is the packet pipeline in more detail. TX and RX threads each handle work queues, transport and congestion control, programmable scheduling, and plane management via firmware and PCIe, while the P4-like engines handle programmable processing.
This figure maps the enhanced RoCE feature set. Multipath sprays packets across up to eight planes. Out-of-order placement handles RDMA responses while keeping sends and atomics in order, and reliable delivery leans on selective ACK and NACK with retransmission.
Path selection runs per packet with headers carrying placement info. SACK bitmaps track received packets per QP. The receiver reorders before in-order message delivery.
Congestion control centers on Receiver Credit-Based Congestion Control as the eRoCE baseline. This receiver hands credits to active senders, uses speculative credits for line-rate startup, and stays aware of ECN, trimming, and CSIG telemetry, with P4-like match-action engines for customizable algorithms. I wonder why we have “P4-like” not just P4 engines.
Now we are at the peer memory path for GPU and XPU systems. Thor Ultra uses dma-buf for local peer-to-peer DMA between the NIC and GPU memory, and eRoCE for remote data transfers, with the RoCE user library in user space and the RoCE driver in the kernel.
Multi-tenancy support includes QoS and virtualization features which seem fairly standard for a modern NIC.
Timing support covers PTPv2 with nanosecond packet timestamping plus Precision Time Measurement. PPS in and out, a time-of-day plane, and an external TCXO feed the PHC here, which is how AI and HPC systems stay synchronized. If you like network timing, we recently filmed a video at a major hyper-scaler’s networking lab, so stay tuned for that in the first half of September.
Security starts at the silicon with a FUSE-based root of trust, a ROM bootstrap loader, and authenticated firmware updates. SPDM device attestation and PSP-based encryption and decryption round out that picture.
Health monitoring, chip and optics temperature tracking, eye diagrams, diagnostics, crash and core dumps, and DMTF- and OCP-standards-based manageability are in the feature set.
Broadcom splits the kernel stack into the bng_re RoCE driver and the bng_en NIC driver, with the libbng_re user library above them, all riding standard libibverbs and rdma-core interfaces.
Peer memory and direct collectives connect to NCCL, RCCL, and MPI through a verbs provider and xCCL plugins, on an upstream dma-buf peer memory model that leaves applications unmodified.
Broadcom shares initial collective data on a Gen5 GPU platform. Across two nodes with 8 Gen5 GPUs each and 16 ranks over sixteen 400G links, the chart puts all_reduce at 383.93 GB/s and reduce_scatter at 380.23 GB/s against a 400 GB/s ceiling, while alltoall is the clear outlier at 84.62 GB/s. That makes sense just given the workloads. Bus bandwidth holds above 96% of line rate across most operations.
On the TCP side, iperf3 on a Gen6 CPU with 512 GB DDR5-8000 hits 791 Gbps unidirectional at 16 parallel flows, about 98.9% of the 800G link. Bidirectional traffic aggregates to roughly 1.51 Tbps, and throughput climbs near-linearly to eight flows before the link saturates. I wonder if we could get this on our Keysight CyPerf bench like we did with the NVIDIA ConnectX-8 C8240 800G Dual 400G NIC Review, where we actually got that running at 800Gbps on a PCIe Gen5 server? That is a question for another day.
For RDMA writes, the unidirectional path saturates at 781 Gbps, and the bidirectional path reaches 1558 Gbps, about 97.6% of the 1.6 Tb/s aggregate and roughly double the unidirectional rate. Unidirectional hits about 88% of line rate by 32 KB messages and lands within 3% of peak by 128 KB.
Broadcom says the hard problems solved in this generation were multipathing, RoCE enhancements such as reliability and out-of-order placement, and programmable congestion control each trace back to the packet processing pipeline.
Here is a look at the Linux tools.
Broadcom packed a lot into this session, from the physical die up through the Linux toolchain. Thor Ultra is clearly built as a systems play for AI scale-out, and the architecture hangs together across silicon, software, and manageability.
Final Words
This Thor Ultra launch puts Broadcom squarely in the 800G NIC contest alongside NVIDIA’s ConnectX-8 and AMD’s Vulcano. Its data path is built for enhanced RoCE or MRC++, and QoS plus virtualization enable XPU and CPU as a service, with security, manageability, and timing for sync across systems.
Given we saw the physical cards and chip almost a year ago at OCP Summit 2025, it was cool to get more detail on the parts.
