Cerebras Wafer Scale Engine Gen2 7nm 2.6T Transistors

4
Cerebras WSE Tesla V100 Side By Side
Cerebras WSE Tesla V100 Side By Side

Just before the global shutdown stopped international travel, I remember a Wednesday in January 2020 very well. I drove over to Cerebras Systems one town over in Los Altos, California to Interview Andrew Feldman the company’s CEO. At Hot Chips 31 (2019) Cerebras was the standout as we covered in Cerebras Wafer Scale Engine AI chip is Largest Ever. I had the opportunity to first see the system built around the WSE, the Cerebras CS-1 Wafer-Scale AI System, at SC19. At Hot Chips 32, Cerebras disclosed some details of its new chip the WSE Gen2.

Cerebras Wafer Scale Engine Gen2

The Cerebras WSE Gen2 offers absolutely huge numbers. For some scale, the Xilinx Versal Premium’s top-end multi-chip package FPGA we covered today, that uses CoWoS to bind together multiple chips, tops out at 92 billion transistors. For GPUs the NVIDIA A100 has around 54 billion transistors. In contrast, the Cerebras WSE Gen2 encompasses 2.6 trillion transistors. That is on a single chip. These are all on 7nm TSMC processes.

Hot Chips 32 Cerebras WSE Gen2
Hot Chips 32 Cerebras Wafer Scale Engine Gen2

This 850 AI optimized core chip is already back at Cerebras headquarters. Likely it is in the same lab we toured earlier this year in Los Altos. This is the only photo we could take from the lab:

Cerebras CS 1 3 Stack In Rack
Cerebras CS 1 3 Stack In Rack

If you are wondering what that is, it is a fully integrated rack of Cerebras CS-1 systems that are being designed for high-density with cooling and power integrated. Cooling was being provided, in this case, not by air, but by facility water, for the specific customer, it was being designed for. That gives one some sense of the demand that Cerebras is seeing. They are beyond the phase of customers looking for single systems and are moving to customers purchasing many systems and asking for higher integration.

Cerebras HQ Q1 2020 Lobby
Cerebras HQ Q1 2020 Lobby

One of the benefits of the giant WSE structure is that it eliminates an extraordinarily amount of chassis, PCB, cooling, networking, and other components from AI clusters.

Final Words

The Cerebras Wafer Scale Engine Gen2 is very exciting. Cerebras declined to comment on the amount of SRAM on-chip, but there is a good chance it has scaled with the number of cores. We expect more information is coming later this year. Hopefully, one of these days I will be able to head over to the Cerebras lab again and see the WSE Gen2 in-person to see if the Cerebras lobby skeleton has managed to thrive through the global pandemic.

The big question we have now is will the CS-1 get a WSE Gen2 upgrade, or will we get a CS-2?

4 COMMENTS

LEAVE A REPLY

Please enter your comment!
Please enter your name here