AMD’s socket G34 platform currently supports up to four eight or twelve core Magny-cours CPUs. While having a dual socket G34 board is interesting, Intel’s 2P platforms dominate most of the x86 DP market at the moment. One thing AMD does to combat Intel on price is specifically in the area of four socket servers. Whereas Intel CPUs must be of the Xeon X series (e.g. the X5670 recently reviewed) or some of the new Xeon E7 series CPUs carrying a hefty premium, AMD uses one set of CPUs across the 2P and 4P G34 platforms. Practically speaking this means one can purchase a 4P motherboard and four low power, twelve core CPUs (48 physical cores in total) for under $3,000. One also gets to access quad channel memory meaning the 4P platform can easily access 32 DIMMs providing good bandwidth and capacity for the platform. TYAN and Supermicro both offer 4P AMD socket G34 lines, and today we will look at the TYAN S8812WGM3NR a quad socket G34 motherboard with an onboard LSI SAS2008 controller.
Recently I acquired two AMD Opteron 6164 HE’s, eight Opteron 6166 HE’s and four Opteron 6174 chips. I will mainly be using the Opteron 6166 HE’s but have also successfully used the Opteron 6174 chips.
- CPUs: 4x AMD Opteron 6166 HE’s
- Motherboard: TYAN S8812WGM3NR
- Memory 16 x 2GB ECC DDR3 UDIMMs
- OS Drive: OCZ Agility 2 120GB
- Enclosure: Norco RPC-470
- Power Supply: Corsair AX850 (850w Gold level power supply)
The main reason that the system only had 32GB of ECC memory was because I wanted to fill all channels and with fourteen G34 CPUs requiring 56 DIMMs to fill memory channels, my ECC memory supply is a bit thin at the moment. Also for those wondering, I used this particular power supply because it was inexpensive, quiet, and efficient. Total power draw at 100% CPU load was around 430w, a very solid figure for a machine of this caliber.
The TYAN S8812WGM3NR is huge! It is officially listed as a MEB form factor measuring 16.2″x13″ (411x330mm). To give one an approximate idea regarding how large this board is, think of a board that is the width of an EATX motherboard but that extends from the 7th expansion slot all the way to the top of an EATX board, and then continues for the width of a power supply. Just to give an idea of how much larger the MEB form factor is versus an EATX form factor I stacked the 2P EATX TYAN S8230GM4NR atop the TYAN S8812WGM3NR just to give a visual comparison:
Given the size of the board, it can be difficult to find appropriate cases. One can either utilize an enclosure with redundant PSUs underneath the motherboard partition or utilize a chassis with a front-mount power supply. I ended up tapping a few holes where the power supply normally goes in a Norco RPC-470 and then re-locating the power supply to the front of the enclosure just to make room.
For expansion slots, the TYAN 8812WGM3NR provides significant expandability despite the fact that it is packed with components. One PCIe 2.0 x16 physical slot can either run at x16 or x8 electrical depending upon whether or not the first x8 physical slot is populated. Aside from that pairing, two additional PCIe 2.0 x8 slots are available. One really nice thing that TYAN includes is the open ended physical slot on all x8 expansion slots. This allows a user to run a x16 card in the x8 physical slot at x8 electrical by hanging half of the card outside of the slot. Since there is precious little room for expansion slots on such a packed PCB, this is a nice feature.
One note on memory, each CPU has eight dedicated memory slots for a total of 32 DDR3 DIMM slots that work in quad channel configurations. That gives one the capacity to run up to half a terabyte (512GB) of RAM in the system.
For drive connectivity, the TYAN 8812WGM3NR does something unusual, it forgoes all but one of the southbridge’s SATA ports and instead uses an integrated LSI SAS2008 based 8-port SAS 2.0/ SATA III 6.0gbps controller. The SAS2008 ports are available in two SFF-8087 ports that run perpendicular to the motherboard’s plane near the bottom edge of the board. While this may sound limiting, the additional PCIe slots and the ability to use SAS expanders with the onboard LSI controller makes it a reasonable configuration.
Network connectivity is provided through three RJ45 connectors linked to an Intel 82574L controller and an Intel 82576EB dual port controller. Other rear I/O panel connectors are basic with two USB ports, a VGA port, serial port. Tyan uses an Intel 82574L controller for the management NIC because vendors like Supermicro use a less expensive Realtek 10/100 NIC for management features. Furthermore, it should be noted that Tyan allows a user the flexibility of using the management NIC as a data NIC. For boards of this class, it is likely one will use an expansion slot for either Infinband, Fibre Channel, or 10GbE connectivity but it is good to have a solid base level of network connectivity. One other item of note is that the board does have a single internal USB header to hold a USB flash drive.
Tyan’s IPMI 2.0 interface will be fairly intuitive if one is accustomed to other modern KVM-over-IP implementations. One can either use external IPMI tools or navigate to a web based interface. I will note that I initially had a few problems with the IPMI 2.0 management NIC not showing up on my network but this was resolved by flashing the BMC BIOS to the newest version. Recently, ServeTheHome.com looked at Tyan’s IPMI WebGUI in-depth.
Being able to remotely manage servers through the IPMI interface is a major time saver. One can troubleshoot common issues, including kernel panics that leave one without access to the machine otherwise, remotely. Even things such as BIOS changes can be accomplished through the web GUI. Users will find standard features such as the ability to control power functions of the server through the baseboard management controller and web interface along with standard features like remote monitoring of fans, temperatures and other sensors.
One of the most important features is the JAVA based KVM-over-IP functionality that allows a user to do several things. First, one can use a keyboard and mouse on a local machine to control the host. Second, one can remotely mount ISOs, local optical drives and etc. onto the server. These features are important to most users unless an external KVM-over-IP solution is used and one has a network switched power distribution unit (or multiple if a redundant PSU is used) that can power cycle a server. Oftentimes using the BMC’s power cycle features are considered more reliable than using the PDU switching at this point.
Overall, Tyan provides the requisite remote management functionality to compete in the MP server market. To any potential buyer, I would strongly suggest getting a board with these features as they have become necessary management tools for machines not acting as primary workstations. In the near future Tyan’s remote management capabilities will be the subject of an in-depth piece on this site in the near future.
Overall, there are not a lot of options with quad socket G34 boards. TYAN has this model which it made flexible enough to accommodate most usage scenarios. The fact of the matter is that this is a very large motherboard which means finding an appropriate chassis can be a challenge. The large feature packed board hits the street at over $800. This seems like a lot at first, but consolidating additional chassis, power supplies, drives, and etc. when using separate machines yields significant cost savings over utilizing many UP 1U servers. One other really cool thing is that unlike the current Intel platforms, AMD’s G34 has a significant upgrade path in Interlagos, the Bulldozer architecture AMD offering hitting the street later this year. With 16 cores and many other improvements, the Interlagos upgrade path will be an attractive feature for many users.