Supermicro X8DTH-6F Motherboard Review Dual LGA 1366 Xeon LSI SAS2008 Dual IOH 7 PCIe Slots IPMI 2.0
The Supermicro X8DTH-6F motherboard is a substantial dual processor (DP) motherboard loaded with on board features and expansion possibilities. Users looking for a low-end server, or a simple 20 drive system accessed over a single Gigabit Ethernet port should avert their eyes because this board is about as feature packed as they come. For those looking to build a very expandable powerhouse of a server, this is probably a good starting point. I get asked about this board and the X8DT6-F (similar board with a single IOH and fewer PCIe slots) at least once a week so when Supermicro approached me about reviewing the board I decided, albeit daunting, I would check the board out.
This board presented a bit of a challenge as I have been accustomed to reviewing single CPU (UP) systems over the past year. I put together a test system for a few reviews I am working on at the moment and posted the build log for this dual processor test bed earlier.
- CPU: Dual Intel Xeon E5606
- Motherboard: Supermicro X8DTH-6F
- Memory: 24GB ECC DDR3 1066MHz DRAM (6x 4GB UDIMMs)
- OS Drive: OCZ Agility 2 120GB
- Additional NICs: Intel EXPX9501AT, Mellanox MHEA28-XTC, and Supermicro AOC-STGN-i2S
- Additional RAID Controllers/ HBAs: LSI 9211-8i, IBM ServeRAID M1015, Areca 1680LP
- Enclosure: Supermicro SC842TQ-665B
- Power Supply: Supermicro 665w (included in the SC842TQ-665B)
While another configuration may have been better attuned better to highly CPU intensive tasks, the Xeon E5606 CPUs are fairly power efficient while allowing full use of the boards expansion capabilities.
Dual LGA 1366 CPU sockets are probably one of the most important features of the X8DTH-6F. With this, one can employ everything from dual core CPUs to six core CPUs, with or without Hyper-Threading. Both the 5500 and 5600 series Xeon processors (Nahalem-EP and Wesmere(-EP) respectively) are supported. At this point in time, Intel’s LGA 1366 processor lineup is very mature so options range from the sub-$200 per CPU range to over $1,500 per CPU with multiple speed and core count gradients in between. One note is that upon setup with a Xeon E5600 series CPU, I had to enable AES-NI features in the BIOS as the attribute was set to “Disabled” by default.
The Supermciro X8DTH-6F includes six 240-pin DDR3 slots per socket supporting up to 48GB of unbuffered ECC DIMMs and 192GB of registered ECC DIMMs. The support for significant amounts of memory, per system board, is one key differentiator of the dual-LGA1366 platform over the single CPU LGA 1155 and 1156 platforms. The X8DTH-6F packs the ability to house dual CPUs for heavy workstation or server loads, and those same loads, be they CAD applications to virtualization applications, generally require more memory to run efficiently. Co-incidentally Windows 7 x64 is currently limited to 192GB of RAM (Server 2008 R2 in contrast, is limited to 2TB), so for some Windows applications, 192GB may be a practical limit a this time.
The next defining board attribute are the dual IOH36 controllers and the seven PCIe 2.0 x8 slots (in x16 physical) they support. Practically speaking, this gives an absolutely monster amount of expansion capabilities for each system since there is ample PCIe bandwidth across the board. For applications where multiple RAID controllers, iSCSI HBAs, 10Gb network controllers and GPUs will be used (for Hyper-V RemoteFX or GPU computing), the seven expansion card setup allows for a lot of flexibility.
One often-overlooked feature is the number of 4-pin PWM fan connectors on motherboards. The X8DTH-6F has a full compliment at eight 4-pin power connectors. This is significant because it allows the BMC functions of the Supermicro motherboard to adjust fan speeds based on temperature changes and load changes. When combined with appropriate 4-pin fans, and low power components, this can greatly reduce the average fan speed of the in-chassis cooling and likewise the overall noise profile of the machine. For many small and medium business users, as well as home environments, noise is a key concern.
An important consideration with the X8DTH-6F is that the board is very large, being an EATX design measuring 12” x 13” in size. For smaller enclosures, or enclosures with multiple fans mounted in a center partition, it is important to ensure that the board will fit in a given chassis. I was able to fit the board in the relatively compact Supermicro SC842TQ-665B chassis very carefully, but there was not an abundance of room. The EATX design is a positive one as a whole because it provides for not just the ample expandability and list of integrated peripherals, but also because it makes the amount of room to work on the system manageable, and keeps all items aligned for good airflow, including the 24-pin ATX and dual 8-pin CPU power connectors.
Finally, regarding the expansion slots, the PCIe slot closest to the CPU socket is very close to that CPU socket. I used relatively large 4U heatsink and fan assemblies and they came close to, but did not interfere with the cards installed in the first PCIe slot. On the other hand, the same slot is subject to length restrictions because one of the DIMM sockets is in-line with the same PCIe slot.
The Supermicro X8DTH-6F has a lot of built in features aside from the expandability that the boards layout affords. It has an Intel 82576 dual port gigabit controller, LSI SAS2008 SAS 2 RAID controller and IPMI 2.0 features including KVM-over-IP.
One intriguing feature of the X8STH-6F is the onboard LSI SAS2008 controller. This controller has the ability to connect eight SAS 2 and SATA III drives running at 6.0gbps and is backwards compatible with older generation drives. As was covered in the previous reviews of the LSI SAS2008 based LSI 9211-8i as well as the Supermicro X8SI6-F, the SAS2008 controller is fairly well supported in major operating systems aside from FreeBSD where the support is not yet in the STABLE-RELEASE. One notable FreeBSD derived ZFS platform, ZFSguru maintained by forum member sub.mesa, now supports the SAS2008 based controllers out-of-the-box, but the current 0.1.7 release is still not as full featured as some other options out there.
Although the controller initially ships with IR RAID firmware supporting RAID 0, 1, and 10 there are two additional options. First, a user can upgrade the controller to unlock RAID 5 capabilities using a RAID key. This option is good for users that want low-cost RAID 5, but for performance striped parity setups, I still recommend a proper RAID controller with a battery or flash-capacitor backed write cache. The other option, and perhaps the one most used, is flashing the controller to IT mode. The primary benefit here is that it allows the controller to present disks directly to the operating system, thereby simplifying the control structure of the disks. It is recommended that users of mdadm and ZFS based systems flash the controller to IT mode to increase reliability. One can note that the LSI SAS2008 controller is in addition to a standard compliment of six SATA II ports (with SPGIO headers) and one internal USB header.
An Intel 82576 dual port Gigabit Ethernet Controller is built in. The 82576 is the same controller used in the Intel ET Dual Port Server Adapter, EF Dual Port Server Adapter, and both the ET and ET2 quad port adapters when used in pairs. This is an update to the 82654L used in many previously reviewed boards such as the ASUS P7F-E, Supermicro X8SI6-F, X8SIL-F, X8ST3-F, and the Intel S3420GPLC. Aside from these two Intel ports, Supermicro also includes a Realtek based NIC for dedicated IPMI 2.0 access.
IPMI Remote Management Features
Supermicro’s IPMI and KVM-over-IP as described a few times on this site, allows for a lot of deployment flexibility. Things such as fan speeds, chassis intrusion sensors, thermal sensors, and etc. can be monitored remotely and alerts setup to notify the administrator of issues. Beyond this, the Winbond WPCM450 BMC chip also allows for the remote power up, power down, and reset of the server in the event that it becomes unresponsive. I will note that this setup has been running 10Gb networking tests and Folding@Home for the past six days without needing to use the below buttons for anything other than turning on the system. In fact, the test system has never had a keyboard, mouse, CD/DVD ROM, or monitor hooked up to it, even after multiple BIOS tweaks and three operating system installations.
Another important feature is the ability to remotely mount CD images and floppy images to the machine over the Realtek Ethernet controller. This keeps maintenance traffic off of the primary Intel NICs and at the same time removes the need for an optical disk to be connected to the X8DTH-6F.
In combination with the baseboard level management features just described, KVM-over-IP features of the X8DTH-6F are not to be underestimated. One can log on to the server using either a web browser with Java platform support, or through Supermicro’s IPMIview software and have remote console capabilities, with mouse support included. Oftentimes, users opine that shell access is enough to troubleshoot, but this BMC level access to the remote system allows one to immediately see things such as a hung and incomplete boot process, frozen servers, as well as manage motherboard and add-in card BIOS remotely. I believe that the IPMI 2.0 features, including the KVM-over-IP features are essential for a server-class motherboard unless one has an external KVM-over-IP unit and a network addressable power source. For users with redundant PSUs, as will be the most common use case with this motherboard, remotely toggling on/ off multiple power supply ports across multiple PDUs can be difficult making the motherboard option a bit more elegant.
The dual LGA1366 with dual IOH36 chipsets platform provides for a lot of innate expandability. With up to seven PCIe 2.0 add-in cards, 192GB of RAM, and fourteen on board disk drive ports, on board dual Gigabit ports, and support for up to twelve physical cores, the Supermicro X8DTH-6F provides a solid base that will be able to expand to support many roles. The other side of the equation is price and at around $570 retail, the board is about $200 more than the X8SI6-F. For that $200, the X8DTH-6F offers a similar feature set with the on board LSI2008, dual Intel GigE ports, IPMI 2.0, and etc., but the X8DTH-6F delivers much more in terms of expandability.