At STH, we have done a number of switch teardowns. Today, we have something special. We are getting to look at a switch that has 32x QSFP-DD ports each capable of running 400GbE bi-directional traffic. In this article, we are going to take a look at the Innovium Teralynx 7 based switch, and then see it pass 12.8Tbps of traffic in both directions. This is the same type of switch being deployed by hyper-scalers today.
As we have been doing in this series, we have an accompanying video:
As always, we urge our readers to open this video on YouTube for a better viewing experience.
Inside an Innovium Teralynx 7-based 32x 400GbE Switch
First a quick bit of background. Innovium is the upstart in the data center switch market directly targeting a segment long dominated by Broadcom Trident and Tomahawk product lines. In terms of success, with the 400GbE generation, Innovium has now shipped over 1M ports and has made sizable inroads in a market where Broadcom had a virtual monopoly.
The switch we are looking at today is based on the Innovium Teralynx 7. This is a 12.8T switch product. Innovium also has the lower-end Teralynx 5 which is more for Top of Rack and other applications where as Teralynx 8 is the next-gen for 800GbE.
In terms of key features, we have Flashlight which is the company’s telemetry and analytics solution. Part of Innovium’s key differentiation is being able to handle this level of analytics alongside its high-performance/ low latency Terascale switch fabric and its Innovlex programmable pipeline.
Recently, Innovium raised $170M in a funding round valuing the company at over $1B due to its portfolio and market success thus far.
With that, it is time to get to the hardware.
Innovium Teralynx 7-based 32x 400GbE Switch Hardware Overview
As is customary, we are splitting our hardware overview into external and then internal sections. Simply due to the nature of video versus still images, you can see more angles in the accompanying video. One will notice that this is a switch that says “Innovium” but it is made by one of the dozens of OEM/ODM partners that Innovium has. We were borrowing the switch, and this is what we could get. We expect most hyper-scalers to not have this paint and that the Taiwanese ODM will not brand their product the Innovium TERASwitch 7.
Innovium Teralynx 7-based 32x 400GbE Switch External Hardware Overview
The 1U switch has its front mostly filled with 32 ports. These are QSFP-DD ports. QSFP-DD is required for 400GbE and is likely to be the standard our readers will see most often.
On one side of the switch, we have the status LEDs and a big reset button. On a relative scale, this is gargantuan compared to many we have seen.
On the right side, we get a management RJ45 port, a USB port, and a serial console port.
Moving to the rear of the switch, we can see that this 1U chassis rear is almost all handles, latches fans, and power connectors.
The fans are, as one would expect, hot swappable. They also have status LEDs to help diagnose a failed fan module.
The power supply we will discuss more around in the power consumption section, but this is a 1.3kW 80Plus Platinum redundant power supply configuration.
After that look at the outside of the switch, next we will take a look inside before getting to power and performance.