Key Lessons Learned
If you have been building servers for years, or if this is for a first build, a motherboard like this is really exciting. The layout is very easy to work on, especially if you use M.2 storage. There is plenty of room around the PCIe slots for dual-width cards like the NVIDIA RTX 6000 Ada or RTX 5000 Ada. Things like the fan headers are all in one place making them easy to find.
Perhaps the bigger lesson, however, is the EPYC 8004. This is absolutely a line that folks are sleeping on in the industry. There are many environments where going up to a 350-400W CPU or dual high-power CPUs is not going to be possible. Siena offers a low-power option with a ton of PCIe Gen5 connectivity. The 64-core maximum and DDR5 help ensure there are enough cores and memory bandwidth when doing two-socket to single-socket consolidation in newer servers.
This is one platform where, if AMD made cloud pricing per core available to the EPYC 8004 series, the space would take off. Siena is really good, but AMD charges a lot for the chips.
Final Words
Overall, the ASRock Rack SIENAD8-2L2T offers a lot of great functionality. Onboard 1GbE, 10Gbase-T, and plenty of expansion is awesome. The fact that ASRock Rack managed to keep this in a 12″ x 9″ ATX form factor while some others have had to use larger motherboards is equally impressive.
It took many little things like pulling an ATX 24-pin power connector but the company even managed to fit two extra DIMM slots for those who do not want a multiple of 6 for memory capacity (although we do have non-binary memory these days.) These little things go a long way in making this platform a great one.
Overall, ASRock Rack delivered the SIENAD8-2L2T almost exactly as we would expect an EPYC 8004 motherboard to look from the company. Lots of utility with a little added flair.
It is indeed a straightforward design. 7 pin SATA means exactly that – 7 pin SATA. Might be my first time seeing that!
As for SATA routing, is it possible that you just give up that noted PCIe slot while using the 7-pin bare SATA headers at the bottom of the board for cabling? Again, I’ve yet to see SATA headers like this.
Looks really interesting to pick up secondhand in a few years. If sold or purchased thru such channels, would the login/password reset to admin/admin?
Hey Patrick! No offense, but where is benchmarks, that one bench chart don”t counts?! Where is power consumption graph, pointless review. Where is comparison table where we can see what features supermicro offers for diy folks , lets say with x13 and Enerald Rapids. Sorry but thats low quality article. Thats my opinion.
and whats the main differences between integrated nics on various platforms lately, some choosses Broadcoms, some x550, some X710.?! Thats worth to deepdive and make seperate article, with power consumption features etc
How much traffic we can push thru discrete nics, lets say, mellanox4, 5 , 6 or , how much Gb /s that various platforms using if all ports populated with u.2 drives?
SERVE THE HOME -right? i hope this site is still for diy folks…..give us numbers, more benchs, less bulshit.
Can we get some power consumption figures for the Siena cpus please
Perhaps there is a breakout board forthcoming for the 16x SATA capable slots. Since this is PCIe it would be cool if they actually supported 12gigabit SAS as well as SATA. The PDF user manual does show them in it’s BIOS description. It also notes the existence of four SATA 7-pin headers but says nothing more about them. Nothing about how the MCIO can be configured.
Interesting board.
SATA can be accessed with a card similar to this (sorry can’t find the specific one): https://www.asrockrack.com/general/productdetail.asp?Model=RB1U2SL#Specifications
Should be 1U2G-RB1U2SL-4G.
Any idle power usage numbers for this motherboard and platform? It’s one of the key features over higher power usage servers!
As far as SATA, the manual (page 7/8) says the onboard headers are SGPIO, which I believe would be for enclosure status LEDs. Page 22 also references a RB1U2SL riser card for SATA (https://www.asrockrack.com/general/productdetail.asp?Model=RB1U2SL#Specifications), though I don’t see it for sale anywhere
@Gasmanc we usually do power consumption on whole systems, instead of just motherboards. In a system like this, figure ~200W for the 64C CPU (Siena stays close to TDP more like Bergamo) but then 30-40W for DDR5, tens of watts for the fans, PSU power loss, and so forth. You end up with 50% more power just to get the system powered on than the motherboard takes, and that would be in a bench design. A server would be more due to additional fans for example.
@laughingman These days most server platforms are thermal limited. It is less like the consumer side where there are huge variances. When we set systems up with the same CPUs and settings, we expect a +/- 3% variance with different CPUs across a range of tests. I am not sure what reviewing NICs not on the motherboard accomplishes. Pushing traffic on a CX-6 is line rate in a platform like this so that is not going to accomplish anything. Where are the U.2 drive bays on this motherboard? We can just make up random metrics, but we try to keep it close to the product being reviewed and what differentiates them, not just random thoughts. Figure 100K+ folks will see this review. I take very seriously trying to stay near the product since that is a lot of people who I do not want to introduce random thoughts and waste time for the majority of them.
If we just want Siena power, we are going to have another server review soon with it.
I was curious on how the SATA ports worked. Page 22 of the user manual has a diagram of a riser that goes into the x16 slot and has two connectors for break out cables for the SATA interface. The 0.1″ pitch headers on the bottom of the motherboard are for SGPIO.
If you search for the riser part number 1U2G-RB1U2SL-4G R1.00 a different ASRock Rack system comes up with what looks like this, or really similar, riser in it.
Still, why put the SATA ports in the slot that needs a riser and not the MCIO connector already on the motherboard? Maybe the connector height would be too tall for a 1U chassis?
Rewording and summsrizing the manual isn’t much of a review. The coverage of anything Siena related has been a summary of the press release or product manual at best.
So this motherboard has two DIMMs on CHD and CHH but the four other channels have only one. While a person could leave the black socket unpopulated, I wonder if the Linux memory allocator makes reasonable choices about memory from the shared channels.
Said another way, the NUMA of EPYC processors is already so complicated that the shared DIMM channels might be more a gimmick than something useful.
While optimising such a memory architecture is likely out of scope, it would be interesting to characterise the applications where populating the black socket made a performance difference. Such results might help satisfy people posting above concerning how they want to hear more than what appears in the manufacturer’s documentation.
@Eric
In the default configuration the allocator has no knowledge of the memory topology, it has a map of valid physical memory addresses and that’s it. It’s the job of memory controllers in the CPU to manage optimal placement.
Software control is possible only with NUMA, and even then it heavily depends on the implementation.
I’d imagine the two dual-DIMM channels are there for use cases requiring more RAM for the price of slight bandwidth reduction (when compared to more single-DIMM channels). Probably ASRockRack had place near the socket to spare, so why not. I’ve seen Intel Supermicro boards doing the same thing.
Admittedly an edge case here, but has a Threadripper or Threadripper Pro CPU been dropped into this board? I am curious what the unofficial interoperability is with Siena and AMD’s workstation parts on a board like this is. (Of course the official support is no but has anyone tried it?) I want to know mainly because AMD is doing some interesting segmentation between these platforms like Siena leveraging Zen4c chiplets for increased density/lower power while Threadrippers use the vanilla Zen 4 chiplets for higher clocks. I suspect that with Zen 5 and Zen 6 on the horizon, AMD will be distinguishing these platforms with differences in V-cache and accelerators.
@Mike, @Mama, @ajr – thanks for the breakout board info. I did look for something like that on asrockrack site but I had no luck.