Chelsio is out with a new DPU line targeting later this year as the world transitions to next-generation server platforms. The Chelsio T7 and D7 DPUs are the first DPU offerings from Chelsio, and we wanted to show what the new DPUs will offer, and what the state of development is.
What is a DPU? Background
At STH we have been covering DPUs a lot over the past year and change. For some background reading and watching:
These mostly have videos as well but they explain what a DPU is, and why it is different from traditional NICs. The AWS Nitro was really the first to implement this and we recently discussed the market in the AMD-Pensando video.
The short version, however, is that a DPU is a processor that combines a few key characteristics. Among them are:
- High-speed networking connectivity (usually multiple 100Gbps-200Gbps interfaces in this generation)
- High-speed packet processing with specific acceleration and often programmable logic (P4/ P4-like is common)
- A CPU core complex (often Arm or MIPS based in this generation)
- Memory controllers (commonly DDR4 but we also see HBM and DDR5 support)
- Accelerators (often for crypto or storage offload)
- PCIe Gen4 lanes (run as either root or endpoints)
- Security and management features (offering a hardware root of trust as an example)
- Runs its own OS separate from a host system (commonly Linux, but the subject of VMware Project Monterey ESXi on Arm as another example)
With that, let us evaluate the Chelsio T7 DPU against our DPU framework to see what the company is offering.
Chelsio T7 DPU Line Launched for 400G Generation
Chelsio specializes in high-performance offload NICs and is commonly used by large storage vendors. Transitioning to a DPU in the next generation means that the company can both upgrade its accelerator IP, and offer a new set of features to its customers. Many large OEMs use Chelsio NICs, so this is a big deal.
The Chelsio T7 is set to be built on TSMC 12nm to get more density as well as ensure supply which has been a challenge in the industry. For those wondering, NICs tend not to be on leading nodes like a 10nm Intel, 7nm/ 5nm TSMC so 12nm is actually quite advanced for this class of device. That leads to lower power consumption which is important as we have seen some DPUs break 75W already. Chelsio has both versions that will have ARM cores and also versions without Arm cores to lower costs.
Onboard the Chelsio T7 DPUs are high-speed handling 25 to 400GbE speeds. These are PCIe Gen5 x16 devices that can be an endpoint or a root. There is a PCIe Gen4 switch, Arm processors along with DDR5 memory. Chelsio has its dataflow engine and traffic manager instead of a programmable P4 pipeline in this generation. Chelsio has its host of co-processor accelerators and even an Ethernet switch onboard. Overall, we are going to say the Chelsio T7 fits into our DPU category.
There is another version, called the Chelsio D7. This is for computational storage applications and can also do things like attaching to FPGAs or other chips while providing a host of services. This goes up to PCIe Gen5 x16 and can act as an endpoint or root as well. We get a host of accelerators and on-chip RAM along with up to 200GbE speeds. We are going to also call this a DPU.
The Chelsio S7 we are actually going to call a high-end 200GbE capable offload NIC. Here we lose features like the ability to act as a PCIe root, we lose the Arm cores, and this is designed to not use external memory to reduce costs. As a result, we are going to say this fits as an offload NIC in the DPU vs SmartNIC and the STH NIC Continuum Framework. That is somewhat the point since Chelsio is differentiating the T7 and S7 based on their respective feature scopes.
On the subject of features, there are a lot. This is a bit of an eye-chart but Chelsio is focused on providing features like RoCE, iWARP, NVMeoF, NVMe/TCP and more all in a single NIC solution. One of Chelsio’s key value propositions is being able to span the large feature set with a single device rather than require OEMs qualify a different device for NVMeoF and NVMe/TCP for example.
Here are some of the applications that can use the new chips. One can see the FPGA example for specific RTL that Chelsio’s customers may want to implement. There are even NVMe to Ethernet bridges mentioned, similar in concept (but scaled up) to the solution powering the Kioxia EM6 NVMeoF SSD we recently showed.
A lot of Chelsio’s features will work with the new T7 generation without major updates, but there are a few items that will change:
My sense talking to Chelsio is that we are still a few months from seeing cards. This makes sense because the PCIe Gen5 servers are still many months away. With that said, Chelsio has a giant Intel Stratix 10 FPGA emulation platform that is designed to map all of the gates of the new chip. The goal is that Chelsio and its big OEM customers can have an exact gate-for-gate platform to do development on between now and when there are chips and cards available.
Please do not confuse these with actual T7 DPUs that will go in systems, but it just looks cool so we decided to keep it in this piece.
Overall, the shift to DPUs is happening. It is great to see Chelsio get into the game. We are going to have a lot of DPU content this quarter and year, but in the meantime, feel free to watch the STH NIC Continuum.
The NIC has come a long way since my days of building servers with T3/T4/T5 cards (+ a couple other vendor’s of the same class.)
The problem with Chelsio is their horribly messy software. It is massive, bundles gazillion of things, and is very hard to use, or integrate with other components (i.e. other vendors cards) in modular way, plus a lot of features usually not in upstream Linux kernel. Which is a shame, because hardware is very capable, has a lot of encryption and storage offload, that are really useful.
Chelsio software has in fact been very solid and in production with multiple tier-1 OEMs for years across several generations. The code is delivered via Linux k.org or FreeBSD kernels and is peer reviewed by the community, the windows variants are all WHQL certified, etc. One of the many offloads that Chelsio’s solution does (kernel mode TCP offload), is not accepted by the Linux OS (but is by other OS’es), and for Linux that is delivered separately. The solution can do many offloads concurrently and as such the various drivers are delivered as part of the package when a drop from the k.org or FreeBSD is not used – the customer does not have to use all of them. We don’t know the circumstances of your particular experience to comment but invite you to get in touch with the company (firstname.lastname@example.org) to address your exact concern. That the solution has been in production for over two decades is a testament to the robustness of the solution and we endeavor to support you to have a good experience.