ASUS ESC4000A-E11 Review 2U 1P AMD EPYC and Instinct GPU Server

6

ASUS ESC4000A-E11 Internal Hardware Overview

Next, we are going to look inside the system. We are going to work from front to rear, already having covered the frontmost portion with the drives and front expansion. We will note that ASUS uses an excellent hard and transparent plastic airflow guide that is better than many in the industry. The company also keeps the area for this airflow guide clear, making it very easy to remove or install.

ASUS ESC4000A E11 Overview Front
ASUS ESC4000A E11 Overview Front

One of the most notable features here is cooling. Inside the system, we get an array of fans. Three fans cool the CPU, power supplies, memory, M.2, center expansion slots, and the rest of the motherboard. On either side, we have two fans that cool the main GPUs, FPGAs, or other cards. All of these fans are hot-swappable. Something different is that the connector is in the center hub of the fan.

ASUS ESC4000A E11 Hot Swap Fan
ASUS ESC4000A E11 Hot Swap Fan

Behind the fan partition, there is the main PCIe and power distribution area. One can see the PCIe cables leading to the front of the chassis. power is distributed to the system and the GPUs via the side edge connectors. Under the sea of black cables, one might be able to spot the M.2 connector close to the fans. There is a single onboard M.2 slot capable of up to M.2 22110 size drives.

ASUS ESC4000A E11 PCIe And Power Distribution Center With M.2
ASUS ESC4000A E11 PCIe And Power Distribution Center With M.2

Unlike the ASUS RS720A-E11-RS24U we reviewed, this system is a single-socket-only platform. Inside we have a single AMD EPYC 7003 “Milan” or EPYC 7002 “Rome” series CPU and SP3 socket. This system can handle up to 280W TDP CPUs thanks to the generous cooling.

ASUS ESC4000A E11 CPU And Memory
ASUS ESC4000A E11 CPU And Memory

The memory situation is interesting. There are a total of 8x DDR4-3200 slots. Our system came installed with two 32GB DIMMs for 64GB, but we would deploy it with 256GB or more of memory and all eight memory channels filled. Luckily we have an extensive spare parts bin.

ASUS ESC4000A E11 CPU And Memory Rear
ASUS ESC4000A E11 CPU And Memory Rear

Behind the CPU is the dual low-profile x16 riser. On one side, we can see the small heatsink for the Intel i350-am2 NIC.

ASUS ESC4000A E11 Rear Riser Left
ASUS ESC4000A E11 Rear Riser Left

On the other side we can see a microSD card slot and an ASPEED AST2600 BMC.

ASUS ESC4000A E11 Rear Riser Right And ASPEED BMC
ASUS ESC4000A E11 Rear Riser Right And ASPEED BMC

The riser slot itself is interesting. ASUS uses two special riser slots in a row. Each is PCIe Gen4 x16 in order to provide connectivity.

ASUS ESC4000A E11 Dual PCIe Gen4 X16 Riser Slots
ASUS ESC4000A E11 Dual PCIe Gen4 X16 Riser Slots

The star of the platform, however, are the two risers on either side of the platform. ASUS has a few different configuration options, but many will use this either as an 8x PCIe Gen4 x8 or 4x PCIe Gen4 x16 platform. Half of those slots are on either side. In both of our risers, we have an AMD Instinct MI200 installed, but one can see the four PCIe slots on each riser.

ASUS ESC4000A E11 AMD Instinct MI200 Riser
ASUS ESC4000A E11 AMD Instinct MI200 Riser

The AMD Instinct MI210 is the lower-power PCIe version of the MI200 family. This is the same GPU family that powers the AMD EPYC and Instinct-based Frontier supercomputer. The Fontier GPUs are OAM form factors so they can be much higher power, but the MI210 is the PCIe variant.

ASUS ESC4000A E11 AMD Instinct MI200 Riser 2
ASUS ESC4000A E11 AMD Instinct MI200 Riser 2

Here are the key specs for the MI210 PCIe GPUs that we have.

Model MI210 PCIe
Compute Units 104 CU
Stream Processors 6,656
Peak FP64/FP32 Matrix 45.3 TF
Peak FP64/FP32 Vector 22.6 TF
Peak FP16/BF16 181.0 TF
Memory Size 64GB HBM2e
Memory Clock 1.6GHz
Memory Bandwidth Up to 1.6 TB/sec
Bus Interface PCIe Gen4
Infinity Fabric Links Up to 3
Max Power 300W TDP

One would be correct in noting that how the system shipped to us with 2x MI210’s but also 2x 32GB DDR4 DIMMs means that we had twice the GPU memory as we had CPU memory in this system.

Next, let us get to management.

6 COMMENTS

  1. It’s a bit unusual to see last-gen servers getting reviewed. Is there a concrete reason why the successor to this model, the E12, wasn’t offered for review? It is already available and comes with Epyc Genoa support.

  2. Hm … Nvidia H100 is PCIe gen 5 – what about config with actual AMD CPU which supports the gen 5 I/O ?

  3. It’s a fantastic server, but you have to know what”s missing too:
    1. There is no PCIe bifurcation option. You can install only one M.2 ssd on board or 4 M.2 ssd if you choose to not have a RAID controller.
    2. You can”t do RAID on default configuration, there is nothing onboard to do that. You have to install a RAID controller.
    3. There is no SATA M.2 on board.
    4. The SDcard is used for BMC log only. You have to to some non supported cobfigurations to boot from SD card.
    5. There is no USB port inside the server.
    6. You can”t do RAID with U.2 disks.
    7. If you install a RAID controller, you have only 2xU.2 disks.

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.