At OCP Regional Summit 2023 in Prague, Celestica showed off an 800G switch. The company told us this is a Broadcom Tomahawk 5 switch and it is using technologies like internal cables to help mitigate the challenges of routing and cooling. This switch was not on the company’s website, so we thought STH readers might want to see this.
Celestica Shows an 800G Broadcom Tomahawk 5 Switch at OCP Regional Summit 2023 Prague
The first thing that may look a little different about this switch is the orientation of the optical cages. Instead of being densely packed and horizontal, these are oriented vertically and with large spacing.
Here is the overview of the switch. We can see a few differences between the 100GbE and 400GbE generations we typically review. The first is that we see a fairly standard management processor with dual M.2’s that are off of the CPU module. The ASPEED AST2600 BMC is notable. Celestica is not using an OCP DC-SCM style format here even though a removable BMC is a key feature of that design.
The switch PCB is interesting as well. While the chip is not marked we were told at the show that this is a Broadcom Tomahawk 5. Since that is an 800GbE capable switch chip, that seems to make sense. This switch chip is not connected to the optical cages via PCB.
Instead, cables are being used. These cables solve two major challenges in 800GbE switch designs. First, they allow for easier trace paths to the pluggable optic cages. With 800Gbps speeds and dense switch trace needs, it is difficult to minimize noise while traversing PCB. One option is co-packaged optics. A lower-cost/ easier option is to use cables.
The other major challenge is something we can see here. The 800G cages need larger heatsinks just to cool the optics. Plus, the cables need landing spaces on both ends. Those are large heatsinks for ToR networking lasers.
An alternative to this approach is co-packaged optics. We are going to start to see that in the 800G generation, but that should be a design that becomes more popular as time goes on and Ethernet speeds double every 18 months or so.
Final Words
Celestica has zero information available on this switch given a quick web search. Still, it was very cool to see what is essentially a next-gen switch on display. We are starting to see a big push to 400GbE even down to the individual server/ node level now that PCIe Gen5 x16 slots can support the bandwidth. We would expect to see the same for 800GbE in 2024/2025 as we start to see PCIe Gen6.
This switch is also quite a bit different from the Celestica Seastone DX010 32x 100GbE Switch we looked at some time ago. Hopefully, we can find one of these to tear apart in the future.
In the meantime, we are wrapping up a Tomahawk 4 64x 400GbE switch piece that should be live in the next two weeks so stay tuned for that.
I have a silly question: according to Broadcom’s website the Tomahawk 5 supports 64×800 GbE ports, but from the pictures looks like the switch only has 32 ports. Where are the rest of the ports?
Maybe they didn’t implement them? Tomahawk 4 is a 400G max switch based on the broadcom site’s specs
This looks like Celestica’s old Tomahawk4G switch which they also showed at OCP October 2022. Same implementation and cage orientation and 32x800G cages (2x400GE each) yields 25.6T (TH4), not 51.2T (TH5)
Could the person who told you about this be wrong?