Dell S5232F-ON Internal Hardware Overview
Inside the switch, we have a fairly standard layout with ports, the switch chip and its PCB, and then the management and power/ cooling.
The switch chip here is the Broadcom Trident 3 X7. It is covered by a fairly large heatsink.
The Broadcom Trident 3 is a 16nm switch chip that is designed for more features while the Tomahawk line is designed for outright performance. Dell also has Tomahawk-based Z series switches in this generation. The Trident 3 family spans the X2, X3, X4, X5, and X7 models ranging from 200Gbps to 3.2Tbps. This switch is built around the X7 3.2Tbps switch chip. We can also see two 10GbE management ports that align with a feature we saw on the front of the switch.
Next to the switch chip, we see a Lattice (LMXO3LF-4300C) FPGA, but then also a small PCB. This board houses a Microsemi ZL30363 (now Microchip) chip. Here we have the same chip, also on a separate PCB, but it is a different board than we saw on the S5148F-ON. That previous generation also had Celestica’s silk screening on it.
This Microsemi ZL30363 is an IEEE 1588 and synchronous ethernet packet clock and dual-channel network synchronizer. Feel free to look it up a bit more, but here is the block diagram for that.
The other side of the PCB is relatively lower density.
Behind the main switch PCB, we have the PCB that handles the power supply inputs, fan modules, and management boards.
Something that is changed is that the CR2032 battery, while still on this main PCB, is now accessible when these management boards are installed.
One of the big changes, aside from the switch chip, is the move to an Intel Atom C3000 series “Denverton” chip. This is a five-year newer processor than was found on the S5100F-ON series. Here is an example from a Supermicro motherboard that we took for the launch of the Atom C3000 series. You can learn more about Denverton on STH. We hosted Denverton Day Official STH Intel Atom C3000 Launch Coverage Central when Intel decided to release the part and not publicize it so we have a massive amount of content on the Atom C3000 series.
In this generation, the SSD storage is now up to 64GB versus 16GB in the previous generation. The 16GB of onboard memory is found via the SODIMM on top of the board but the second SODIMM is actually below the management PCB.
Another huge change is that the S5232F-ON has a standard BMC. While Dell EMC uses a proprietary iDRAC controller on its PowerEdge line, some non-PowerEdge servers, as well as the vast majority of servers outside of the big 3 legacy server vendors Dell, HPE, and Lenovo use the ASPEED BMCs. Large hyperscalers use the ASPEED parts not just in servers but also switches, storage, arrays, JBOFs, JBOGs, and so forth.
Something really interesting here is that technically this makes the Dell EMC Networking a “very high-performance network focused Intel Atom C3000 series system with an industry-standard ASPEED BMC.” We still wish Dell had a firewall appliance or an edge server with this setup but without the Broadcom chip.
Next, we are going to look at the software implications of this switch’s hardware.
I love that you do these.
Interesting article – thank you. I was a bit confused by the 124x 25G part, since the switch spec sheet specifically states support for 128x 25G.
Esteben:
Max 25GbE density 124 (QSFP28 breakout)
Source: S5200-ON Spec Sheet
It is a typo in the table. In the text in the spec sheet it says:
In addition to 100GbE Spine/Leaf deployments, the S5232F-ON can also be used in high density deployments using breakout cables to achieve up to 128 10GbE or 128 25GbE ports.
Also in older spec sheets it lists the same 128 25GbE port.
We always go by spec tables if there is no clear abundance of documentation otherwise. The 10GbE ports are listed as 124 in the table and 128 in the text as well. Dell has had years to produce correct documentation and we have to assume the table is correct since that is what most people read and has the most specs.
I’d like to see the 96 port version you’ve mentioned in the video.
Did you by any chance get serial-over-lan to work on the BMC. I’ve tried this at one point, bit never succeeded. Also, Dell support and others further in the organization was not of much help. It seems like something that should just work.
Did you get sonic into a working state? One of my colleagues didn’t find it easy to get ports configured with the open version and gave up in the end.