In our HPE ProLiant DL325 Gen10 review, we discussed how we now run many of the servers in the STH infrastructure. We generally buy systems with 24 core AMD EPYC chips. Paired with 8x 32GB DIMMs each, that gives us just about 10GB of RAM per core plus a bit for system overhead. We finally received our first AMD EPYC 7002 generation HPE ProLiant DL325 Gen10 and there was a major change to the networking side that we do not think is for the better. The new HPE ProLiant DL325 Gen10 that just arrived lost it’s signature basic networking which impacts the entire system.
First Generation HPE ProLiant DL325 Gen10 BCM5719 Networking
Here is the test HPE ProLiant DL325 Gen10 we used for our AMD EPYC 7002 generation review. You can see the four network ports that sit between the VGA and iLO port.
Here is one of the first systems we purchased looking down on that section. Between the HPE SAS mezzanine on the left, and the PCIe riser on the right you can see the Broadcom BCM5719. This is a quad-port 1GbE networking option that is integrated into the motherboard. HPE calls this the HPE 1 Gb 331i Ethernet adapter solution.
For some perspective here, this Broadcom chip was populated not just on every ProLiant DL325 Gen10 we installed all the way down to the ~$1100 low-end EPYC 7351P system we bought. It is also present in the HPE ProLiant DL385 Gen10 we have in the lab. To us, this seemed like a standard feature.
This “standard” feature of 4x 1GbE has been changed with the latest server that arrived in the lab.
Enter the HPE Ethernet 1Gb 4-port 366FLR Adapter
On the newest system, we saw the spec that said 4-port 1GbE and thought that it would be the same Broadcom configuration. This server was ordered just before the DL325 Gen10 review went live on STH. When it arrived a few weeks later, this is what we saw:
Apologies for the photo. Datacenter lighting. What you will notice in the upper middle of the photo, is that the Broadcom BCM5719 was de-populated as were the 4x 1GbE ports. Instead, a quad 1GbE NIC is installed in the FlexLOM slot.
From HPE’s current Quick Specs: “NOTE: Embedded 4x 1GbE LOM is not equipped on new boards for 7xx2 series processors.”
The adapter used instead is a HPE Ethernet 1Gb 4-port 366FLR adapter. On this FlexLOM NIC is an Intel i350 controller which is a perfectly well-supported NIC. Still, it is different.
The change has two impacts. First, if you are an existing HPE ProLiant DL325 Gen10 or DL385 Gen10 customer, you need to be extra careful when ordering your HPE ProLiant DL325 Gen10 systems now, especially pre-configured models. Older systems (e.g. the EPYC 7001 series HPE P04648-B21) will most likely utilize HPE 1 Gb 331i Ethernet adapter. The refreshed versions (e.g. the EPYC 7002 series P16696-B21), especially the Smart Buy configs are likely to have the Intel NIC.
Perhaps more profound is that this now utilizes the FlexLOM slot for lower-value 1GbE networking. We have been using 40GbE HPE FlexLOM adapters in this slot. With the new systems, replacing the HPE 366FLR 1GbE card means that the server no longer has 1GbE networking that we use for hypervisor management and provisioning networks. To get higher-value networking the option is now to add a PCIe network adapter instead of purchasing a FlexLOM upgrade. Effectively that means if you want higher-speed networking, the newer HPE ProLiant DL325 Gen10’s with the Broadcom NIC de-populated has a single available PCIe slot instead of two.
For those that rely solely on 1GbE networking, and have no Broadcom v. Intel preference, this is a complete non-issue. On the other hand, for our lab use, this change makes the systems no longer suitable. As we were installing this in the lab and opened the box to find this configuration, we had a few folks in the data center look. Unanimously this design was not the favored solution instead of having consistent onboard networking since very few shops are running 1GbE for anything other than management networks.
We are, as a result, going to update our HPE ProLiant DL325 Gen10 review to link to this article. HPE discloses this in their spec sheets so it is not an underhanded change by any means. The Intel i350 is a very popular NIC. On the other hand, losing continuity with the HPE 331i / Broadcom adapter from previous systems as well as unnecessarily filling the FlexLOM slot to achieve the same purpose is something we want our readers to be aware of as it has a drastic impact on expandability.
We have an official response from HPE on this and wanted to share with our readers:
HPE is committed to delivering latest technologies and the best experiences to customers. We introduced new levels of choice and flexibility in network adapters for the HPE ProLiant servers based on customer feedback.
The 4 x 1GbE network interface (LOM), standard on the HPE ProLiant DL325 and HPE ProLiant DL385 Gen 10 servers with the 1st Gen AMD EPYC™ (Naples), was removed from the subsequent variants equipped with the 2nd Gen AMD EPYC™ processors (Rome) that we introduced in August. This allows customers to choose from 1GbE, 10GbE and 25GbE interconnect options without worrying about isolated and unutilized resources.
If a 4 x 1GbE NIC experience is required, two choices are available: an HPE Ethernet 1Gb 4-port FLR Adapter in a FLexibleLOM form factor or an HPE Ethernet 1Gb 4-port PCIe Adapter in a stand-up form factor. Both will deliver the same tested and trusted experience as the now discontinued LOM.
Additional information on the HPE ProLiant DL325 Gen Server, based on 2nd Gen AMD EPYC, can be found here: https://h20195.www2.hpe.com/v2/getdocument.aspx?docname=a00045498enw (Source: HPE)