The ASRock Rack GENOAD8X-2T/BCM might be an uncomfortably good motherboard. It has a huge number of features that those looking to build an AMD EPYC-based system will want, exposing the platform’s huge I/O capabilities to builders. At the same time, this is a platform with some quirks to it, so it is a fun one to review. Let us get to it.
ASRock Rack GENOAD8X-2T/BCM
Let us start of with the most important spec of the motherboard, it is EEB (12.63″ x 13″) in size. That means it will fit many larger cases on the market today, but it will require looking for EEB support. Even with a size that is much larger than a standard ATX motherboard, this system is absolutely packed with functionality to the point it almost feels like ASRock Rack ran out of space at some points. Many server motherboards today use proprietary form factors, but boards like this help those who wish to build their own servers in specific chassis.
Let us start with a prime example of what makes this platform cool, but also the constraints it is under. We get an AMD socket SP5. Just for some sense, that allows for up to a 96-core AMD EPYC 9004 “Genoa”, a 96-core with 1.1GB of L3 cache Genoa-X, or a 128-core Bergamo. Putting it succinctly, one can get roughly the performance of a dual-socket previous-generation server in this single socket.
There are trade-offs to fit these massive CPUs. One of the most prominent is memory. This motherboard only supports 8-channel DDR5 memory in one DIMM per channel (1DPC) mode. We recently explained Why 2 DIMMs Per Channel Will Matter Less in Servers but we are losing 50% of the memory channels and 66% of the maximum capacity with this design. On the other hand, with DDR5 we get roughly 50% more memory bandwidth than the previous generation’s 8-channel DDR5 designs. Still, it is less than the CPUs are capable of in terms of performance and capacity. Putting 24 DDR5 DIMM slots next to a SP5 socket would have taken up an enormous amount of motherboard space so the trade-off was made.
Something else that is a bit different with this platform is the power input. There are three ATX12V CPU power inputs along with a standard ATX power input. Having CPUs that can use 350W or more plus a huge set of components means these systems need more power. We started STH in the era when it was common to have a single ATX12V power connector onboard.
Then comes the expansion slot arry that many will be excited to see. There are seven PCIe Gen5 x16 slots and one x8 slot. Four of those x16 slots can run in CXL 1.1 mode.
Seven PCIe Gen5 x16 and one x8 slot would be a lot of I/O alone, but there is more, a lot more. An example is the dual M.2 slots onboard.
Before we get to the rest of the I/O, we should probably address the chips found all over this motherboard. There are Phison PS7101 chips all over which are PCIe Gen5 signal redrivers with mux/ demux. With PCIe Gen5, even reaching signals from one side of the motherboard to the other can be challenging from a signal integrity standpoint, and even moreso if, for example, a motherboard is designed like this one where the two PCIe x8 segments on the AMD Green2 side can drive one PCIe Gen5 x16 slot or send x8 to the slot and x4 to a M.2 and x4 to a MCIO connector. Those different distances were not as big of a challenge in the PCIe Gen3 era, but with PCIe Gen5, it is a signal integrity challenge, hence why chips like the Phison PS7101 are being used.
As one can see, there are two MCIO connectors on the opposite side of the motherboard from the PCIe slots with the M.2 slots in the middle.
That white 6-pin connector is a graphics 12v power connector.
There are two more MCIO x4 connectors above the PS7101 array and between the DIMM slots and the front panel USB 3 header and fan header array.
These MCIO connectors are the options for connecting not just slotted PCIe cards but also NVMe SSDs and with some of the connectors things like SATA SSDs. We are going to put the block diagram in this review because it is a very complex setup.
On the subject of the rear I/O something we wanted to quickly mention is that the airflow of this motherboard is standard server front-to-back. Some will make pedestal servers out of this platform, but we strongly advise that if you are looking to use the top vents for CPU exhaust, that you do so with liquid cooling to not interfere with the system airflow.
At this point, we probably have more than a handful of STH readers thinking that this would make an amazing workstation platform. Those are excellent thoughts, however, one of the big challenges is the enormous gap between the server and workstation rear I/O. This platform has a decidedly server rear I/O with legacy serial, VGA, and two USB ports for local KVM cart access in the data center.
The single network port stacked atop the USB ports is for out-of-band management. The motherboard has an ASPEED AST2600 Baseboard Management Controller or BMC onboard. This runs an industry-standard management stack for IPMI and functionality like iKVM access.
The two network ports are 10Gbase-T ports powered by the Broadcom BCM57416 controller that sits under a heatsink behind the VGA and serial ports. This is also the reason for “BCM” in the GENOA8X-2T/BCM product name.
This is a complex motherboard, so next, let us get to the topology.