A Look at the FS N9510-64D A 64x 400GbE Switch

9

FS N9510-64D 64-port 400GbE Switch Internal Overview

Here is an overview of the switch internals. We are going to do something a bit different this time and work from the rear (left below) to the front (right below).

FS N9510 64D 64 Port 400GbE Switch Internal Overview
FS N9510 64D 64 Port 400GbE Switch Internal Overview

Here is a quick look at one of the hot-swap fan connectors.

FS N9510 64D 64 Port 400GbE Switch Internal Fan Module Connector
FS N9510 64D 64 Port 400GbE Switch Internal Fan Module Connector

In this switch, there are two levels of fan control boards each powering four fan modules.

FS N9510 64D 64 Port 400GbE Switch 2 Level Fan 2
FS N9510 64D 64 Port 400GbE Switch 2 Level Fan 2

Inside the switch, there are two main baseboards.

FS N9510 64D 64 Port 400GbE Switch 2 PCB
FS N9510 64D 64 Port 400GbE Switch 2 PCB

The middle baseboard has the Intel Xeon D-1627 (4 core/ 8 thread) management processor. Next to it is a single M.2 SSD for OS storage but there is a second unpopulated slot.

FS N9510 64D 64 Port 400GbE Switch Management CPU
FS N9510 64D 64 Port 400GbE Switch Management CPU

Moving up to the main switch and QSFP-DD PCB, we can see Lattice has some placements here.

FS N9510 64D 64 Port 400GbE Switch Lattice
FS N9510 64D 64 Port 400GbE Switch Lattice

There is also a Xilinx AMD Artix-7 FPGA.

FS N9510 64D 64 Port 400GbE Switch Xilinx AMD Artix 7 2
FS N9510 64D 64 Port 400GbE Switch Xilinx AMD Artix 7 2

The big feature on this PCB is the massive heatsink that covers the main switch chip.

FS N9510 64D 64 Port 400GbE Switch Internal Heatsink
FS N9510 64D 64 Port 400GbE Switch Internal Heatsink

The main switch chip is the Broadcom Tomahawk 4. What is fascinating here is that this switch chip is socketed in a TE Connectivity LGA8371. We covered this solution more in our This is a Broadcom Tomahawk 4 64-port 400GbE Switch Chip piece.

Broadcom Tomahawk 4 BCM56990B0KFLGG In TE LGA8371 4
Broadcom Tomahawk 4 BCM56990B0KFLGG In TE LGA8371 4

As a quick one here, the QSFP-DD cages above can be seen with large cage heatsinks. That is one of the big differences between QSFP-DD and OSFP since OSFP usually has the heatsinks on the optical modules/ DACs.

Here is the 4th Gen Intel Xeon Scalable, codenamed “Sapphire Rapids” next to the Broadcom Tomahawk 4.

4th Gen Intel Xeon Scalable Sapphire Rapids Next To Broadcom Tomahawk 4
4th Gen Intel Xeon Scalable Sapphire Rapids Next To Broadcom Tomahawk 4

Just for some sense, here is the Dynatron J12 for the AMD EPYC 9004 “Genoa” next to the FS Tomahawk 4 heatsink.

AMD EPYC Genoa Vs Broadcom Tomahawk 4 Heatsinks 1
AMD EPYC Genoa Vs Broadcom Tomahawk 4 Heatsinks 1

Internal grounding seems significant between the PCBs.

FS N9510 64D 64 Port 400GbE Switch Board To Board
FS N9510 64D 64 Port 400GbE Switch Board To Board

Here is another photo we had of two connecting boards.

FS N9510 64D 64 Port 400GbE Switch 1
FS N9510 64D 64 Port 400GbE Switch 1

Next, let us get to putting together the network and getting everything working.

9 COMMENTS

  1. That heatsink is crazy. Makes me wonder what is going to happen (en mass) first: co-packaged optics, or water-cooled switches?
    Question: How much does the Tomahawk 4 weigh?

  2. Just musing..but is it odd a 64 port 400GbE switch (and other enterprise or service provider level equipment) is presented on a site called “Serve the HOME” Don’t get me wrong, I love the content here!

  3. Yeah, that is an interesting switch. The thing is so big that it needs a standard server chassis just for that MASSIVE heatsink and 8,000-plus pin Broadcom Tomahawk 4 switch ASIC. Very cool. 🙂

  4. Page 1, paragraph 4: “The top 4U is used for cooling.” should read “The top 2U is used for cooling.”

    Page 3, paragraph 3: “We will quickly note that while we were using Intel Xeon “Sapphire Rapids”” – missing end of sentence.

  5. Sure hope this has a better warranty than “30 days then we send it back to China for a few months” I experienced with an obviously much less expensive switch.

  6. as mentioned before, use the 2x200G cx-7 nic and a cheap ($150) QSFP-DD to 2 x QSFP56 DAC.
    Don’t waste your time with the flat top OSFP weirdness!

  7. @Kwisatz: It’s definitely a misleading site name and it causes so much confusion it’s an FAQ in the “about” section: https://www.servethehome.com/about/

    “STH may say “home” in the title which confuses some. Home is actually a reference to a users [sic] /home/ directory in Linux. We scale from those looking to have in home or office servers all the way up to some of the largest hosting organizations in the world.”

    I do wonder how many of us ended up here because we were after information about home server equipment. I guess judging by the number of non-enterprise mini-PC and 2.5 GbE articles, quite a few of us.

  8. I wonder how often 400Gb ports are saturated in this context – aren’t latency and rate more interesting?

  9. It would be useful if you would cover the management features of the switch. Sure, the heatsink is cool and all but that’s not really functional or useful to users, is it?

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.