ASUS ESC8000A-E11 Internal Hardware Overview
We are generally going to go through the system from front to back, and show some key items along the way, and we are going to pause around the fans since that is a big story here.
Before we get too far, we did want to mention a few items since there are many configuration options here. For example, this is a stock photo of the machine without a PIKE SAS card in front, but a dual M.2 slot option instead. Also, on the GPU side of this chassis, we have the eight PCIe Gen4 x16 slots and can see their cables with fans for the bottom expansion cards underneath. What is different here is that one can see that ASUS has covers between the top and bottom sets of fans to blank off slots when a card is not installed. This helps ensure there is airflow if one has fewer than eight double-width GPUs. We did not have these blanks but wanted to show them.
In terms of CPUs, there are two AMD EPYC CPUs along with a full set of DIMM slots between the front section and the fan partition. These sit on the bottom U of the chassis while the GPUs occupy the top 3U of the 4U chassis.
Along with the two AMD EPYC CPUs, we get 32x DIMM slots. We have 16x 32GB DIMMs here, but one can fit up to 32x 256GB of memory for 8TB total.
The system supports dual AMD EPYC 7003 CPUs. If you recall, on the front bottom of the system, there were two fans on each side. These fans have ducting to direct airflow over each CPU and are secured with a thumbscrew retention gasket. This design looked strange at first, but it is actually a very unique design that we ended up liking.
The impact of this is that the system can cool not just eight GPUs, but also two 280W TDP AMD EPYC 7003 Milan CPUs. Some systems have been limited to 225W or 205W TDP CPUs in these dense GPU servers, but this cooling design helps keep these CPUs cool with a 1U heatsink. That is also important because it allows for more airflow to the GPU area by keeping the CPU heatsink shorter and cooled only via the dual fan inlet portion of the front of the chassis.
In terms of features, you may not expect to see, there is a quite hard-to-access MicroSD card slot on the motherboard behind the DIMMs.
All of the PCIe slots, both for the GPUs and also for NICs/ storage are handled via cabling. So behind the CPUs and memory, we have a large number of PCIe cable headers.
ASUS actually partitions off the chassis into a bottom 1U and a top 3U just behind the CPUs and memory. If one recalls from the overview picture above, the rear PCIe slots are cooled via fans that sit in this bottom 1U of space after some cool air is mixed with the warmed air from the CPUs. That is a very interesting design.
It also leaves four fans to cool the GPUs and a fifth fan primarily cooling the power supplies, but also adding some GPU cooling.
These fans are hot-swappable. It may seem like a small design point, but cooling in these systems is a big deal. Using this 1U on the bottom for non GPUs, and the top 3U mostly for GPUs allows ASUS to use a single 3U row of fans to cool the GPUs. We often see fan assemblies in this class of system that have two fans stacked atop each other extending through the 4U of the chassis. This is a truly interesting design.
These fans push air through GPUs. Each of the GPUs has a retention plate that both secures the GPU with the third point of contact as well as ducts air from the fans through passive GPU heatsinks.
Power is provided by a pair of power cables. One can change these cables depending on the GPUs that one uses. A nice feature is that by being on this bar at the top of the chassis, these are very easy to swap out.
Here we have eight GPUs installed. One will notice there is very little room between them.
These are NVIDIA A40 GPUs. As 300W 48GB Ampere GPUs, they would have been part of the Quadro line in years past. Now, they are simply marketed as the NVIDIA A40, but that is why these are passively cooled GPUs with display outputs.
We did not have NVLink bridges, but one could, in theory, get bridges to connect GPUs via NVLink here instead of just using PCIe.
Next, let us get to management.
Wow. What a system!!!
eth hashrate?
Very interesting system Patrick. Thanks for making this article about it. It is rather impressive to see this amount of GPU and CPU power, along with full DIMM capacity. It is not clear from the ASUS website or the article, but it appears that this system also repurposes one xGMI link, like the R7525, to reach 160 PCIe lanes.
The screenshots from the ASPEED AST2600 iKVM look exactly like the Lenovo SR655.
The choicest web hosting as a replacement for small and good province | Hostupp
If you want to start a web business and use cobweb hosting championing your website, then earliest of all, you have to be au courant of the net hosting services. You should be acquainted with what cobweb hosting is before choosing it as your preferred maintenance provider. If you are looking in behalf of the most suitable trap hosting service in UK that can help you step on it an online self-assurance in prospective, Hostupp is the conservative preference for you. With our web server optimization tools and premium network applications and plugins, we deliver a one and only spider’s web suffer and bear higher conversions with our perspicacious features at affordable prices. We believe in sacrifice scalable solutions without any compromise on quality or deposit through providing 24/7 person support. If you are looking to convert your story into reality, you contain happen to the sound place.
The Best Hosting Serving in 2021. That is not our words; this is powwow of mouth. We had the most satisfied customers from round 30 countries in the domain who are enjoying our network play and drug demonstrative customer strengthen services since starting operations. Start your stoop proceed promoting success with Hostupp now!
Super solidly speed, uncontrolled bandwidth, importance apps and plugins
While comparing to other hosting companies terminated the web, you will find them claiming their server speeds as highest practical ones but when it comes to physical results; they ease up behind a extensive way. With our ultra fast data processing features, dedicated hardware organization and SSD servers we provide a blazingly fast messenger worry hurriedness on all your pleasure ranging from static websites to online stores built with wordpress or magento ecommerce stage, hosting wordpress blogs and wordpress website visitors.
We are Hostupp – The later of snare hosting
What is the purpose of the SD card slot? Would you use it as an OS drive?