We have a review of a motherboard we have featured on STH before, and you may have seen in a few projects. Specifically, we have a review of the ASUS Pro WS WRX80E-SAGE SE WiFi. This is an AMD Ryzen Threadripper Pro (or what we call a “WEPYC“) motherboard that tiptoes the line between a workstation and a server motherboard. Indeed, it is also one we did something uniquely special with, and hopefully, that gets folks excited.
ASUS Pro WS WRX80E-SAGE SE WiFi Hardware Overview
The Pro WS WRX80E-SAGE SE WiFi is a large motherboard. It is listed as “EATX” but at 309.8mm x 330.2mm. The standard EATX is 305mm so this is notably wider. Let us just call it how it is though, this is an impressive motherboard that every moment you touch it you can tell it is a big step up from a standard consumer motherboard. ASUS is marketing this as a high-end workstation board, pairing it with the Threadripper Pro, and this feels the part.
Since this is an AMD Ryzen Threadripper Pro motherboard, we get the AMD socket SP3 that the company uses for its high-end CPUs. That socket is flanked by heatsinks and DIMMs on all sides.
In the socket, one can use up to 64 core/ 128 thread CPUs such as the AMD Ryzen Threadripper PRO 3995WX. We have also looked at the 3975WX 32-core and 3955WX 16-core processors. One of the biggest benefits here is that the CPUs can use DDR4-3200 ECC RDIMMs with eight-channel memory. This adds a huge amount of memory capacity and bandwidth to the solution.
Just as a quick note here, cooling can be a challenge. Many of these SKUs, like the 3995WX that may be paired with this board, have a 280W TDP. That practically means that using large air coolers one will have to find a server-oriented front-to-back cooler that also fits around all of the heatsinks and DIMMs.
Practically, many will end up using liquid cooling with this platform. That allows heat to be transported away as well as access to components. An air cooler that can cool 280W TDP will obstruct access to the DIMM slots making them difficult to service. This front-to-back airflow design along with the high TDP means we expect many systems with this motherboard to be liquid cooled.
In terms of expansion, this is clearly a high point of the platform. The AMD Ryzen Threadripper Pro series benefits from having massive PCIe Gen4 connectivity between the CPU and the WRX80 chipset.
Here we get seven PCIe Gen4 x16 slots.
These slots allow for multiple GPUs, NICs, storage devices, and accelerators to be installed. One small but nice feature is that the bottom edge’s connectors are angled to be horizontal to the motherboard allowing for cabling without interference from the bottom slot’s device and heatsink.
For storage, we get a lot. Here we get eight SATA ports and two U.2 ports on the edge of the motherboard. Four of these SATA ports are from the chipset, four are from an ASMedia ASM1061 controller. The two U.2 ports shared bandwidth with the onboard M.2 so one has to decide if they want to use M.2 or U.2. Also, one of the U.2 slots originates from the WRX80 chipset.
Just as a quick note here, the horizontal connectors are great, but one does have to be careful in terms of the clearance in the chassis. The Fractal Design Define 7 XL is a large chassis, but the ASUS motherboard is so large that it can create connector clearance issues just due to some of the case angles.
The top left of the motherboard has many power connectors for the CPU, PCIe power, and so forth. These are not even the only power connectors as there are two 6-pin PCIe power connectors at the bottom of this motherboard.
These power connectors are vertical, so they create a bit of a wall in front of the CPU socket if cables are not managed properly. If you do air cool this system, that is something to be very cognizant of.
ASUS also has a Q-LED for its post-code checking feature and a front-panel USB 3.2 Gen2 connector.
Getting to the M.2 storage situation, there are a total of three PCIe Gen4 x4 M.2 slots. Two do share lanes with the U.2 ports, but it is nice to have an option to use either without an adapter.
These M.2 slots go around the WRX80 cooling solution that includes an active fan. Over time, just be aware that this is another fan to clean and keep dust away from.
Beyond the storage, power, and front panel USB connectors, we should quickly call out that this platform has power and reset buttons on the board itself.
In terms of rear I/O, there is a lot. First, we get rear panel clear CMOS and BIOS flashing buttons. We then get WiFi 6 that is provided by an Intel AX200. This platform was just before the WiFi 6E modules started to make it to the market, and upgrading is not overly easy to do. On the networking side, we also get dual 10Gbase-T using an Intel X550-t2 NIC. That is a high-end server solution. One can also use this NIC as a shared NIC for the ASUS ASMB9-iKVM IPMI management solution, but there is not a dedicated NIC port. There is also a full set of audio connectors.
In terms of USB, there is certainly a lot. We get a total of eight USB 3.2 Gen2 Type-A ports and two Type-C ports for ten total. A major reason to purchase this workstation platform instead of trying to make a server into a workstation is this focus on USB and audio connectivity.
Also, a feature we always like is that the rear faceplate is affixed to the motherboard making it slightly easier to install.
Next, we wanted to highlight a few other hardware features before getting to the management aspects.
Additional Hardware Features
First off, the ASUS board is absolutely huge and feels heavy. We normally do not feature box shots, but this just has an awesome presentation in its very heavy box.
The back of the motherboard is certainly interesting. We get a backplate more akin to what one might expect on a GPU rather than a motherboard. This helps add rigidity and also to prevent shorts on the back side of the motherboard. The systems that have these units are so complex that it requires a lot of time to disassemble if there was something shorting on the back of the motherboard.
In terms of storage, we also get a Hyper M.2 card. This has cooling plus additional PCIe Gen4 M.2 slots. A quick note here is that it is taller than the PCIe bracket so you will not be able to put this in a 3U height case. ASUS basically assumes you will use this system in a larger workstation chassis, and that makes sense given the system profile.
Here is a look of the card without the cooling shroud. One can see the four Samsung 980 Pro NVMe SSDs installed on the card and the radial fan. There is no PCIe switch, so this card relies on the PCIe bifurcation on the motherboard. That is not an issue since this is a server derived platform.
In terms of memory, this system can support 2TB, and especially with the larger 32 core and 64 core parts we would expect to see 256GB, 512GB, and larger configurations being common.
As a quick note here, we are going to suggest a large chassis again for this platform. We are showing this system with the Fractal Design Define 7 XL which is a large chassis, and there were still some tight spots and bits we had to move in the configurable chassis to make everything work. Given how much connectivity this platform has, it is probably wise to go a bit bigger rather than trying to find a small chassis to match, especially if you are using all of the platform’s connectivity features.
One item we are a bit bummed about is that we could not find a block diagram for the motherboard. Given all of the connectivity and the WRX80 chipset, it would have been a nice feature to have. If anyone finds this, please feel free to send it our way and we will update.
Next, let us look at management.
ASUS Pro WS WRX80E-SAGE SE WiFi Management
For management, the BIOS is very interesting. We get features from the consumer side, but then with a few additional server features. One can see, for example, we have a server management option in this workstation platform because the Threadripper Pro is a thinly veiled EPYC platform.
Some of those workstation features are nice, such as the ASUS EZ Flash 3 utility that is better than many server motherboards. One could argue that server motherboards would just use IPMI, and that is fair, but these are small nice to have features that come from having both server and workstation BIOS.
On the server management side, we get BMC support for the onboard ASPEED BMC. We can also see status and do standard BIOS-BMC configuration tasks.
One also gets the full ASUS ASMB9-iKVM solution that we see on system like the ASUS RS720A-E11-RS24U we review with the AMD EPYC 7763 and NVIDIA A100. that also means we get features such as a HTML5 iKVM that we can see here on the same screen as the BIOS shot above.
One can also use this for tasks such as setting up an OS remotely. Something that we actually used this system for, and we will get to this next, is as a GPU-less workstation and that is where this feature shines.
Overall, it is great to have a workstation with this feature. It makes remote management easier. At the same time, if one wants to transition this workstation to server duties for remote rendering or as another server platform, having out-of-band management is great.
Next, let us get to the use case for something like this that is a bit out of the ordinary.
Notes:
1. Cool setup.
2. You went the GPU/Mellanox(NIC) setup. A Storage/Mellanox(NIC) setup is similarly interesting. WEPYC is also destined to outclass most professional storage solutions in network and storage bandwidth.
3. In addition that platform is also ideal for high-end KVM/Hyper-V/Container platforms. In fact this is the first time, where a terminal server can outclass any workstation in CAD/CAM, GPU, gaming, parallel processing. Tags: GPU-P in Hyper-V, PCIe cut-through,…
4. The article sounds “slightly” one-sided. From our testing the Supermicro M12SWA-TF is slightly faster.
Nice read.
Curious how you got fan control on your motherboard. I’ve had one for about a week, have a support ticket in with Asus, but no response. Cpu is 100% all the time.
I have issues with GPUs, I run 4x 3090.
My power supply is 1000W for 2x 3090 and 1600W For System and 2 GPUs.
First of all, GPUs are plug-in slots 1-2-3-4 not as recommended in slots 1-3-5-7 (when I use the recommended, the system won’t boot)
When I use slots 1-2-3-4 I can’t shut down the system at all. Some GPUs kick out (red lamps on ROG STRIX) and the powerplay won’t stop at all.
Can anyone help.
Fan controls are over BMC not BIOS on this motherboard, read the manual.
I bought a new Motherboard a week ago along with the 3955wx… Inserted RAM chips 1 by 1 starting DIMM_A, B, C, D….H.
The motherboard screamed at me with 1 long and 2 short beeps when I inserted sticks in the A and D slots (one by one as well as together); in all the other slots, it POSTed just fine with 00 Code.
I felt bad to torture the RAM slots till I was convinced that the RAM slots were either dead or had some issues. I even removed and inserted the CPU, checked all the pins of the CPU socket…. it was all clean.
Now the board is under RMA… I will surely update once I receive the new one.
Cheers guys.