AIC HA401-TU Compute Nodes
Here is the side view of the compute node stack outside of the chassis. From what we can tell, these nodes are identical.
Each node has two CPUs, memory, NICs, boot drives, and power delivery paths. One notable design choice is that the fans are on the sleds instead of being part of the chassis. That means that the fans can be serviced by pulling one of the two redundant controllers and the remaining controller and drives can remain in the rack.
Here is a look at the rear of the nodes. As one can see, there are a lot of cables, including Ethernet cables.
Taking a quick look here we can see a M3808AK controller for a LSI SAS3808 generation of PCIe Gen4 x8 SAS3 controller.
That connects into one of the most unique cards you will see. Here starting from the right we have two PCIe 8-lane connectors that go into a board with an OCP NIC 3.0 slot. In that slot, we have a dual port 10Gbase-T controller. That dual port OCP NIC 3.0 controller has a mezzanine attachment for the SAS3 controller that sits on top.
That 10GbE is part of the HA solution, but here is a look at this card which absolutely should wow you. This setup is absolutely our favorite part of the server.
These three layers sit atop the Broadcom SAS expander board.
Since we only have 8 ports with this stack we also need a Broadcom 35X48 SAS expander on each node to go from the 8 controller ports to the 24 drives on the backplane. We can see more SFF-8643 connectors for the SAS expander on the PCB here.
Behind the wall of fans, we get the CPU and memory. In the system, we get dual 3rd Gen Intel Xeon Scalable processor sockets (Ice Lake) each with 8x DDR4 DIMM slots.
Here you can see we are using Micron memory. Luckily, we had additional Micron DDR4-3200 RDIMMs and Intel Xeon CPUs to add to this chassis since a full configuration across both nodes takes four CPUs and 32 DDR4 RDIMMs.
Although there are the 10Gbase-T NICs, and SAS3 infrastructure at the front of the chassis for storage and networking, that is not the only networking involved. Instead, there are a pair of M.2 slots inside, again with a Micron M.2 SSD.
One can also see the PCIe slots. We get a total of 3x PCIe Gen4 x16 slots, 2x x8 slots for networking, AI accelerators (we tried the NVIDIA L4 and that had enough airflow), and external SAS HBAs.
Here we can also see our ASPEED baseboard management controller. AIC uses an American Megatrends MegaRAC solution for its management software providing an industry-standard management experience via IPMI.
Above the PCH, we also get the OCP NIC 3.0 slot for even more connectivity. This is likely the primary networking for a system like this.
An advantage of a SAS3 chassis and dual CPUs is that we get plenty of PCIe lanes.
One other, feature is that we get 2.5″ drive bays on the rear. AIC has options for either PCIe/ NVMe or SATA drives. Our system is configured for NVMe as we can see from the big cable here.
These rear 2.5″ bays have nice tool-less drive trays as well.
With all that is going on, the next step is that we want to look at the block diagram. There is a LOT going on here so it is important to see how this entire setup works.