Supermicro X10SRL-F Review – UP Xeon E5 V3 Motherboard
The Supermicro X10SRL-F review is our first UP motherboard review of the Haswell-EP/ Intel Xeon E5-2600 V3 generation. With the increased core counts, 10 SATA 3 ports onboard and a transition to DDR4, there is a fundamental switch in the capability of new UP servers and usability of the arrangement as building blocks. Furthermore, with chips like the Intel Xeon E5-2650L V3 one can fit 1U servers with 12 cores and 128GB+ RAM in a 1A @ 120V power limit given the right PSUs and non-AVX workloads making this segment even more interesting.
We used a fairly typical UP setup for this type of server and what we have used previously in our Intel Xeon E5-2600 V3 coverage. It should be noted that the Intel Xeon E5-1600 V3 line-up is compelling for those looking for fewer cores and higher clock speeds.
- CPU: Intel Xeon E5-2690 V3
- Motherboard: Supermicro X10SRL-F
- Memory: 8x SK. Hynix 16GB DDR4 2133MHz ECC RDIMMs
- SSD: 2x Samsung 840 Pro 256GB 4x Crucial M500 240GB, 4x Samsung SM843 480GB
- Operating System: Ubuntu 14.04 LTS, Windows Server 2012 R2, VMware ESXi 5.5
Overall, the 128GB configuration is not as dense as one can get with larger 32GB DDR4 RDIMMs and larger LRDIMMs, however it is still a reasonably priced configuration.O verall this is a simple setup but should give us a good idea regarding potential performance. We did use our standard add-in card suite including the Mellanox ConnectX-2 and ConnectX-3 cards, Intel 10 gigabit LAN cards, LSI SAS/ RAID controllers and etc.
Supermicro X10SRL-F Overview
The Supermicro X10SRL-F is an ATX sized motherboard, making it capable of fitting into even many short-depth enclosures. The LGA2011-3 socket sits between two sets of four DDR4 RDIMM sockets for a total of 8 DIMM sockets. One can also clearly see that the socket is a narrow ILM type. See here for a guide to square versus narrow ILM. The airflow is set to be standard front to back configuration.
Probably the biggest feature of the X10SRL-F is its PCIe configuration. Grantley has 40x PCIe 3.0 lanes standard per processor. One can see a total of seven PCIe slots, five x8 physical and two x16 physical. The two x16 physical slots are PCIe 3.0 x8 electrical. There are two PCIe 3.0 x8 physical and electrical slots. Two additional slots are x8 phsyical and can run either in PCIe 3.0 x4 + x4 or x8 + x0 electrical modes. That does give some flexibility to the motherboard and this capability is auto switched based on which slots are populated. Finally there is a PCIe 2.0 x4 electrical in an x8 physical slot. The net effect here is a UP motherboard with 40x PCIe 3.0 lanes exposed and 4x PCIe 2.0 lanes. That is the maximum offered by the platform.
From the block diagram, we can see where the additional PCIe 2.0 lanes from the Wellsburg PCH go, and that is mainly to the Intel i210AT NICs and the AST2400 BMC. The ASpeed AST2400 BMC seems to be more or less standard on this generation of motherboards.
The 6 fan headers onboard are 4-pin PWM fan headers which save quite a bit of power by modulating fan speed to match cooling needs. With the new Haswell-E and Haswell-EP lines, idle power consumption is so low that using PWM fan functions can save significant amounts of power. Along those lines, power is delivered to the motherboard via a standard 24-pin ATX connector and an 8-pin CPU auxiliary power connector.
In line with the modernization of the platform, the X10SRL-F includes internal USB 3.0 headers. Both a Type-A style connector for internal USB keys but also a USB 3.0 front panel connector. The trend toward modernizing the rest of the platform is significant with Grantley compared to Ivy Bridge-EP.
Supermicro includes a total of ten SATA 3 ports. These ports have maximum throughput of 6.0gbps. On the previous generation for Intel Xeon E5 V1 and V2 platforms, there were only two SATA 3 ports. What this practically means is that many 1U and 2U enclosures will not require an additional 8-port HBA to attach a full compliment of SATA drives at 6.0gbps speeds. These extra ports can be a significant source of power savings since HBA’s often can consume ~10w each.
The two yellow ports have corresponding SATADOM power connectors. SATADOMs are very popular in many server and network appliances so one can use the X10SRL-F to quickly attach these modules and corresponding power.
The rear I/O is also significantly upgraded. There are standard VGA and legacy serial ports. The two Springville based i210AT NIC sit next to one another. The third NIC is a Realtek based NIC where the RJ-45 port sits atop two USB 2.0 connectors and is dedicated to IPMI 2.0 Out-of-Band (OOB) management.
Finally there are two USB 3.0 ports on the rear of the chassis. Whereas with the V1 and V2 generation, the primary focus was on USB 2.0 ports, with the Grantley platform we are seeing a majority of ports use the significantly faster USB 3.0 specification. It should be noted that the rear I/O follows a similar layout to the X9 generation of motherboards, so re-use of chassis and I/O covers should be possible in many cases.
Overall the X10SRL-F is a great motherboard for those looking either for a simple compute node or an expandable UP platform. The motherboard is designed using the latest generation components and exposes all of the PCIe lanes possible to the user via expansion slots.
To capture these images we utilize our FLIR Ex series professional thermal imaging camera and turn on FLIR’s MSX enhancements so we can see components outlined clearly. We put the system under 100% CPU load for a period of 24 hours to let “heat soak” set in prior to taking the images.
As one can see, there are some warmer parts to the X10SRL-F but that is relative. There are no real “hot spots” that might cause concern and all components appear to be well below their operational thresholds.
Software and Management
Supermicro’s IPMI and KVM-over-IP as described a few times on this site, allows for a lot of deployment flexibility. Things such as fan speeds, chassis intrusion sensors, thermal sensors, and etc. can be monitored remotely. Alerts can be setup to notify the admins of issues. Beyond this, the functionality also allows for remote power control. You can do remote power up, power down, and reset of the server in the event that it becomes unresponsive. In fact, the test system has never had a keyboard, mouse, CD/DVD ROM, or monitor hooked up to it, even after multiple BIOS tweaks and operating system/ hypervisor installations.
Another important feature is the ability to remotely mount CD images and floppy images to the machine over the dedicated management Ethernet controller. This keeps maintenance traffic off of the primary Intel NICs. At the same time it removes the need for an optical disk to be connected to the Supermicro motherboard.
The latest revision of Supermicro’s management features include some key changes and new functionality. This includes the ability to recover from a bad BIOS flash from the IPMI interface.
We will have more on this feature soon, but suffice to say this makes management of remotely located servers significantly easier with some safeguard in the event a BIOS flash goes poorly.
Overall the X10SRL-F is a very interesting base platform. It offers equivalent functionality to many previous generation UP and DP servers in a UP platform. It also has its PCIe 3.0 and 2.0 lanes exposed to make it a solid building block for highly customized UP servers. Fitting in a standard ATX size motherboard design means it is also flexible and can fit into most standard enclosures. Overall, it is a solid building block for those who do not need many integrated components.