About a month ago we published a piece showing how trying to install VMware ESXi 6.5 on AMD Ryzen and getting a PSOD. Since then, we confirmed that AMD Ryzen plus Windows 10 will yield a usable VMware Workstation 12 Pro setup. Now we have the “fix” to both the VMware PSOD and many other Linux kernel issues: disabling SMT. This is frankly not a good fix. Disabling SMT will yield a 30% (or more) digression in performance. That means that your 8-core Ryzen 7 with SMT off will often be out-performed by a 6-core Ryzen 5 with SMT on. At our STH, our stance is this. If a fundamental feature (e.g. one of the top 5 specs every CPU vendor touts) cannot be used in conjunction with an OS from installation to stable operation, we do not call that working.
Alas, we are going to show how to get something at least running.
Setting Up AMD Ryzen with VMware ESXi 6.5
We took a fairly simple test system and got everything ready to install ESXi. Here is the basic configuration:
- CPU: AMD Ryzen 7 1700X
- Motherboard: ASUS Prime B350-Plus
- Cooler: Noctua NH-L9x65 SE-AM4
- 64GB G-Skill DDR4 (4x 16GB)
- SSDs: 16GB Kingston m.2 2242, Intel DC S3700 800GB, Intel DC S3700 100GB
- PCIe Networking: Intel XL710-QDA2 (dual 40GbE) and Intel Desktop Adapter 82572EI (1GbE)
- Video: EVGA GeForce 710 2GB
- KVM: Lantronix Spider external KVM
We knew the ASUS motherboard’s Realtek NIC was not ideal for VMware so we added the XL710-QDA2 40GbE adapter as this setup in our DemoEval lab and various software companies have used it to patch their distributions at this point.
Here is the video where we get the AMD Ryzen 7 system to install VMware ESXi 6.5 past the PSOD we received in the previous iteration.
Again, as a major caveat, doing this will have significant detriments to the AMD Ryzen 7 system performance to the tune of 30%. While we expect RHEL, CentOS, Debian and Ubuntu systems to be patched soon for Ryzen, VMware is likely a longer timeframe until we expect it will work.
Major Caveats to Getting ESXi 6.5 Working with Ryzen
Here are some caveats you should be aware of if you want to try this.
- Not all BIOS versions have had options to turn SMT off. For example, the ASUS Prime B350-Plus we are using did not get a BIOS update with this option until mid-to-late March 2017.
- As of this writing, Ryzen platforms do not have iKVM or IPMI so remotely deploying them is still not advisable. We, therefore, do not recommend Ryzen systems as “servers” in remote data centers. We have a solution that has been working for the half dozen machines in DemoEval but it is far from ideal.
- Many motherboards will use unsupported NICs. Our test system above has an Intel XL710-QDA2 40GbE card installed. Otherwise, ESXi would not find a supported NIC and fail the installation process.
- Just because something can be installed that does not mean it is ready for stable operation. Ryzen outside of Windows is still not stable enough to be deployed in data centers.
- We know a number of folks are interested in Ryzen ECC support. Now that we have hosted almost three dozen different parties, and with ten test systems set up, we are confident in saying that outside of Windows, the stability impact of ECC memory is negligible. There are still unpatched software issues that will take your system down well before a potential memory error will.
- Given the above, consider this an AMD enthusiast only workaround, and we do not recommend it.