The Most Important Server of 2022 The Gigabyte Ampere Altra Max and NVIDIA A100

3

Gigabyte G242-P32 Motherboard and Ampere Altra Max CPU

The motherboard inside this system is the Gigabyte MP32-AR0. This is a more standard form factor motherboard that Gigabyte built the system around. The first feature many will have noticed by now are the two M.2 slots for boot devices.

Gigabyte G242 R32 M.2 Slots
Gigabyte G242 R32 M.2 Slots

These sit between the DIMMs and the rear GPU area with the PCIe slots behind them.

Gigabyte G242 R32 Barebones PCIe Slots SAS3008
Gigabyte G242 P32 Barebones PCIe Slots SAS3008

Now for the server’s “party trick”. If you want to get to the CPU socket and most of the DIMM slots, you need to lift the power supply portion up. This locks in place to make it easier to service.

Gigabyte G242 R32 PSU Area Lifted
Gigabyte G242 P32 PSU Area Lifted

Here is the server layout fully configured with the power supply section erect.

Gigabyte G242 R32 Overview PSUs Out
Gigabyte G242 P32 Overview PSUs Out

Here is the CPU side of that.

Gigabyte G242 R32 Rear Area Under PSUs
Gigabyte G242 P32 Rear Area Under PSUs

Here is the CPU socket area with its cover. That airflow guide was not my favorite part of the system. If the server looks a bit worn, it arrived to us as a barebones, but one that has seen a lot already in its lifetime.

Gigabyte G242 R32 DIMMs And Socket Barebones 1
Gigabyte G242 P32 DIMMs And Socket Barebones 1

Here is another angle of this area.

Gigabyte G242 R32 DIMMs And Socket Barebones 2
Gigabyte G242 P32 DIMMs And Socket Barebones 2

For DIMMs, we used 8x 64GB of DDR4-3200. We used these DIMMs for a few reasons. First, we wanted to fill the memory channels. Second, we wanted 512GB as it looks like the NVIDIA HPC dev kit comes with. Finally, these are the same DIMMs I brought to Canada when I did theĀ How Liquid Cooling Servers Works with Gigabyte and CoolIT piece. They have been in Intel, AMD, and now Ampere servers. Some folks do not know that modern Arm servers use the same DIMMs as Intel/ AMD. We used these specifically to prove that is possible.

64GB SK Hynix DDR4 3200 RDIMMs 1
64GB SK Hynix DDR4 3200 RDIMMs 1

Ampere’s socket is huge.

Gigabyte G242 R32 DIMMs And Socket Empty 1
Gigabyte G242 P32 DIMMs And Socket Empty 1

The socket itself has five screws that feel akin to an AMD SP3 socket, but without the slotting feature that AMD uses (likely a collaboration with HPE as AMD’s socket design feels like an evolution from the old HPE Xeon E5 Proliant retention.)

Gigabyte G242 R32 DIMMs And Socket Empty 2
Gigabyte G242 P32 DIMMs And Socket Empty 2

This is a gratuitous socket photo.

Gigabyte G242 R32 Ampere Altra And Ampere Altra Max Socket 1
Gigabyte G242 P32 Ampere Altra And Ampere Altra Max Socket 1

Here is another.

Gigabyte G242 R32 Ampere Altra And Ampere Altra Max Socket 3
Gigabyte G242 P32 Ampere Altra And Ampere Altra Max Socket 3

Installing the Ampere Altra Max 128 core processor is something that was unpleasant. Normally one is supposed to use suction cups that we did not have. Instead, I free-handed this one. You can see the installation in the accompnaying video. I have done a few Altra/ Altra Max installations but my advice is to have someone else do it. This is a big CPU package.

Gigabyte G242 R32 Ampere Altra Max With 8x 64GB DDR4 3200 Installed 1
Gigabyte G242 P32 Ampere Altra Max With 8x 64GB DDR4 3200 Installed 1

Here is a look at the Ampere Altra Max 128 core processor, eight 64GB DIMMs, and the two NVIDIA A100 80GB PCIe GPUs installed in the system.

Gigabyte G242 R32 Ampere Altra Max With 8x 64GB DDR4 3200 Installed 2
Gigabyte G242 P32 Ampere Altra Max With 8x 64GB DDR4 3200 Installed 2

There is another fun feature you may have seen. This platform has an OCP NIC 2.0 mezzanine slot. Although we mostly see OCP NIC 3.0 these days, OCP NIC 2.0 is popular because the cards are ultra inexpensive.

Gigabyte G242 R32 Rear OCP NIC 2.0 Slot
Gigabyte G242 P32 Rear OCP NIC 2.0 Slot

Now that we have the system running, we are going to get to that experience.

3 COMMENTS

  1. I guess I’m curious as to what makes the Ampere Altra processor so “unpleasant” to install. Why would installing this chip be any different from installing any other, Patrick? Apart from the physical size, there doesn’t appear to be anything much different between this processor and all the other Intel, AMD and IBM chips I’ve seen.

  2. Stephen, super easy to answer. AMD and Intel both use carriers. AMD SP3 uses the CPU carrier that you slide in, then it is lowered in place. On the Intel side, if you install the CPUs correctly, then you are not touching the CPU when it is lowered aligned into the socket. With Ampere, you need to align it by hand versus AMD and Intel both have mechanical alignment. Even with suction cups, the Ampere socket does not have the same level of mechanical alignment before it is lowred into the socket.

    See how to install LGA4189 correctly Article and Video.

  3. @Patrick Kennedy

    Thanks for the very fast reply. Sounds like a possible engineering/design problem with the type of socket, maybe the ILM needs to be re-designed to allow for the alignment features you mention. Otherwise, I still don’t see how this is an absolute deal-breaker. The chip package itself should have keying notches in the edges that ought to provide enough positive feedback even when loaded by hand.

    Believe it or not, I did see your LGA-4189 installation article when it came out. Very well done and very useful. Now, if only we could have some higher-resolution photos of the chips you feature. I noticed that the photos of the Ponte Vecchio chip you had earlier were only a mere 800×600-ish pixels. Seems we need an upgrade to at least 1920×1080 minimum. I have a hard time seeing much detail, specifically where contact pads are concerned. I like drawing pictures of these things and being able to see the pads well enough to count them is a must.

    After all the recent coverage of the Altra and Altra Max, I’m kind of wishing someone on eBay would start selling used ones so I can buy one. CPU collecting is one of my hobbies and I’m longing to get my first LGA-4189 Xeon, LGA-4094 Threadripper/Epyc and one of these Altra chips. I can’t afford to buy these things when new and typically have to wait at least ten years before they’ll even show up in the used listings. Of course, somebody could donate one if they wanted to, though that’s probably asking too much of the Universe.

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.