Recently, we moved one of our Arista 32x 100GbE lab switches. Patrick, our Editor-in-Chief, pulled it out of the rack and wanted to do a little feature on it, so we put together a short “inside the switch” look of the Arista DCS-7060CX-32S 32x 100GbE switch. Unlike servers that we get to do weekly looks at hardware, switches are much harder to take a look at. As a result, Patrick did the video, and I am doing a web overview so our readers can see how these are made.
Inside the Arista DCS-7060CX-32S 32x 100GbE Switch
If you want to see the video version, you can see it here.
As always, we suggest opening it in a YouTube tab. There are more angles there than we can show in pictures.
Arista 7060CX-32S Hardware Overview
The switch itself is a 1U form factor for standard 19″ racks. The front of the switch has 32x 100GbE ports. One will notice that there are also two SFP+ ports on the left side of the switch.
We also have a management, RJ45 COM port, and USB port on the side to manage the switch.
The rear of the switch has four hot-swappable fans and two power supplies. One can see these have red handles so we have a port to PSU airflow switch.
While the external part of the switch one can see often, we wanted to show what is inside.
Behind the QSFP cages, we have the Broadcom Tomahawk (1) switch ASIC in the middle of the switching PCB. This is a 6.4T switch chip for bi-directional 32x 100GbE port traffic and is capable of around 3.3Bpps.
We will note that Broadcom has iterated on this original Tomahawk chip several times but when it first arrived in late 2015/ early 2016 this was a high-end 1U switch platform.
The switch cooling solution is interesting. There is a center section heatsink with a few fins. That center heatsink has heat pipes that dart to either side with much denser fin arrays. The total heatsink length is around 15″ including the tips of the heat pipes.
The heatsink wings are positioned well above the PCB. Underneath, we can see that there are lower-power components. Here we have an Intel-Altera Cyclone FPGA just below one of the wings.
Behind the main switch PCB we get the control plane PCB. Here is a look at the switch from the rear with the power supplies and fans installed.
Here is another look without the power supplies.
Many of the switches we see have power supplies that connect to the control PCB rather than the switch PCB. This Arista switch is designed to directly connect the power supplies to the higher-power switch PCB.
Powering the system we get an AMD embedded SoC. This is the AMD GX-424CC from what we can see in the switch. Arista has an x86 platform for managing the switch. A fun fact is that this chip technically has Radeon 5E graphics onboard. Although there are no display outputs, technically there is AMD GPU IP in this switch.
Something else that is very important here is that by using an AMD embedded CPU instead of the common Intel Atom C2000 series. As a result, we do not have to worry about the Intel Atom C2000 Series Bug (AVR54). We have had the Intel Atom C2000 AVR54 Bug Strike STH. Atom 2000 switches, without the fix or new chip stepping can suddenly fail to boot. Using the AMD embedded CPU, this particular bug is not a concern unlike many switches such as the Celestica Seastone DX010 32x 100GbE Switch we looked at a few weeks ago.
Memory is very interesting. There are two DDR3 slots and one is populated. Per the spec sheet, we were expecting to see a 4GB DIMM here. Instead, this is clearly an 8GB DIMM.
Another small feature that is not populated here is an M.2 SSD slot. We did not see this in the spec sheet, but it appears to be designed to take up to an 80mm SSD.
The fans have nice hot-swap connectors where tension from red clips is used to keep the fans in place and guide them to the connector.
The power supplies were another area where the switch did not match the spec sheet. The spec sheet says 500W power supplies however these are 495W 80Plus Platinum Delta power supplies.
In the lab, this switch runs under 200W for us and the specs state a maximum power draw of 333W, so to us, the 5W difference is more of a curiosity rather than a cause for concern.
One other quick note that we wanted to mention about this switch. If you want to run EOS, then that is fairly straightforward on this platform. If you want to venture into the world of open networking, then this switch supports SONiC. You can check out Arista SONiC on GitHub.
We took a snapshot of the list above because at STH we had a guide years ago on getting started with 40GbE SDN with Microsoft Azure SONiC for under $1K using the 7050QX 40GBE generation of this part.
Overall, we wanted to show one of Arista’s 100GbE switches that you can use to get started with SONiC. If you either currently or hope to, work in the networking space, it is likely wise to learn about SONiC since that is a huge force in the industry. Also, 100GbE is a rapidly growing standard that will be bolstered by the transition to PCIe Gen4 this year and PCIe Gen5 in 2022 and 2023. In the last year or so, our test platforms at STH have been adopting 25GbE and we use 100GbE switches with breakout cables in our lab racks for testing. With PCIe Gen4 we can get dual 100GbE ports in a single PCIe Gen4 x16 slot, and there are new lower-cost 100GbE NICs such as the Supermicro AOC-S100GC-i2C 100GbE Intel 800 Series NIC hitting the market.
Hopefully, you enjoyed this quick look at the Arista 7060CX-32S.