ASUS ESC4000A-E11 Management
The ASUS management solution is built upon the ASPEED AST2600 BMC running MegaRAC SP-X. ASUS claims this is faster to boot, but we did not time it. The AST2600 is a jump in Arm compute resources, so that would make sense.
ASUS calls this solution the ASMB10-iKVM, which has its IPMI, WebGUI, and Redfish management for the platform. The last EPYC GPU server we looked at from ASUS, the ESC4000A-E10 was still using ASMB9 and the AST2500, so this is an upgrade to a newer faster BMC.
Some of the unique features come down to what is included. For example, ASUS allows remote BIOS and firmware updates via this web GUI as standard. Supermicro charges extra for the BIOS update feature.
Another nice example is that we get the HTML5 iKVM functionality included with the solution. There are serial-over-LAN and Java iKVM options too. Companies like Dell EMC, HPE, and Lenovo charge extra for this functionality, and sometimes quite a bit extra.
ASUS supports Redfish APIs for management as well. Something that we do not get is full BIOS control in the web GUI as we get in some solutions such as iDRAC, iLO, and XClarity. For GPU servers where one may need to set features like the above 4G encoding setting with certain configurations, this is a handy feature to have.
Another small feature that is different from ASUS is the IPMI Hardware Monitor or IPMI HWM. While one can pull hardware monitoring data from the IPMI interface and some from the BIOS, ASUS has a small app to do this from the BIOS.
We like ASUS’s overall approach to management. It aligns with many industry standards and also has some nice customizations.
Something we did want to mention quickly here is that ASUS confirmed that it does not vendor-lock EPYC CPUs using AMD PSB as Dell EMC and Lenovo do.
We like that ASUS is not using this AMD EPYC feature and think that it makes the systems more flexible than those from Dell EMC and Lenovo.
Next, we are going to take a look at performance.
ASUS ESC4000A-E11 Performance
First, we wanted to see how well the CPU cooling works in this system. Some GPU servers do not adequately cool GPUs. As a result, we put in our AMD EPYC 7763 at 280W TDP and saw how it performed.
Overall, we got a really good result here. We did not get a chance to try the AMD EPYC 7773X Milan-X in this system, and that may be a better option. Still, this was fairly great performance. We also think the EPYC 7713P may be a popular option, given its lower single-socket pricing while still offering 64 cores. In any case, it appears as though the ASUS server was able to cool this adequately.
We ran through a few test cases using our HPL scripts on the MI210 and some other GPUs we had data for. There are probably more optimizations available, but here is what we got:
The AMD Instinct MI210 is a Linpack monster. As ROCm continues to improve, other applications will take advantage of the performance. Still, it is well-known that AMD has a very high-end linpack architecture in this generation. Unfortunately, we did not have the NVIDIA H100 to add in here since those GPUs are hard to get.
Next, let us get to power consumption, the STH Server Spider, a word on the MI210, and more.
What is the SD card slot for?
@Matt H.
That’s for the Hypervisors
It’s a bit unusual to see last-gen servers getting reviewed. Is there a concrete reason why the successor to this model, the E12, wasn’t offered for review? It is already available and comes with Epyc Genoa support.
Hm … Nvidia H100 is PCIe gen 5 – what about config with actual AMD CPU which supports the gen 5 I/O ?
Tomas – We have a Gen5 AMD EPYC Genoa server with video going live in a few hours.
It’s a fantastic server, but you have to know what”s missing too:
1. There is no PCIe bifurcation option. You can install only one M.2 ssd on board or 4 M.2 ssd if you choose to not have a RAID controller.
2. You can”t do RAID on default configuration, there is nothing onboard to do that. You have to install a RAID controller.
3. There is no SATA M.2 on board.
4. The SDcard is used for BMC log only. You have to to some non supported cobfigurations to boot from SD card.
5. There is no USB port inside the server.
6. You can”t do RAID with U.2 disks.
7. If you install a RAID controller, you have only 2xU.2 disks.