At Hot Chips 2023, Lightelligence showed off its Hummingbird low-latency optical connection engine. We managed to find one outside the auditorium to take a snapshot of for a cover image.
Still, this is being written in real-time so please excuse typos.
Lightelligence Hummingbird Low-Latency Optical Connection Engine
This is yet another table saying that AI compute demand is far outpacing transistor scaling.
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_02](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_02-800x450.jpg)
Lightelligence is focused on increasing optics in compute. Today, the company is talking about the Optical NOC. This is using a test chip and card that we saw outside, but that is only a part of the company’s portfolio.
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_03](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_03-800x450.jpg)
The company is saying that creating domain-specific architectures to address specific numeric formats, math, and parallelism is one area of performance improvement.
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_04](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_04-800x450.jpg)
Another area for increasing performance is adding more silicon to packages.
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_05](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_05-800x450.jpg)
The interconnects using electrical signals are seen as inefficient.
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_06](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_06-800x450.jpg)
Here Lightelligence is talking about an Optical Network-on-Chip or oNOC. The idea is the use light instead of electrical signaling on a package to increase efficiency.
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_07](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_07-800x450.jpg)
It also allows for different types of topologies since the optical waveguides can span longer distances.
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_08](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_08-800x450.jpg)
It allows better scaling as the distance between chiplets increases. That is important since chiplets can be lower cost to manufacture, but then they need to be connected on packages.
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_09](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_09-800x450.jpg)
Hummingbird is the example of using the oNOC. It has a FPGA, an external laser, and a 3rd party SiP packaged onto a card. That oNOC allows for things like all-to-all broadcast.
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_10](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_10-800x450.jpg)
Hummingbird is focused on putting optical connectivity on a package using a photonic integrated circuit or PIC along with electronic silicon chips.
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_11](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_11-800x450.jpg)
The interposer provides power to the chips. The bottom PIC is the optical component. The EIC is again on the top. That SiP is SIMD architecture with a custom ISA with clusters of eight cores.
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_12](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_12-800x452.jpg)
The big difference is the optical broadcast network with the U-shaped structure. Using that, every core can update every other core and not have any waveguide crossings in the PIC.
Here is the microarchitecture of the core. This is an AI DSA inference core. Every other core can send its data via the oNOC receive and also each core can be sent data via the oNOC quantize and transmit.
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_13](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_13-800x450.jpg)
Here are the metrics on the Hummingbird.
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_14](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_14-800x450.jpg)
Here is a system with Hummingbird in a PCIe GPU chassis.
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_15](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_15-800x450.jpg)
Here are the performance metrics.
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_16](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_16-800x450.jpg)
Lightelligence says it can stitch different sizes of chips. It seems like it is focused on building something beyond a rectile limited chip like the NVIDIA H100.
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_17](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_17-800x450.jpg)
Here are the solutions again:
![Lightelligence Hummingbird Low Latency Optical Connection Engine_Page_18](https://www.servethehome.com/wp-content/uploads/2023/08/Lightelligence-Hummingbird-Low-Latency-Optical-Connection-Engine_Page_18-800x450.jpg)
Final Words
Utilizing 3D packaging will make this technology more interesting. The next question, of course, is whether other companies will look for this for inter-chip communication, or if other types of technologies will take over. This is an interesting technology, but many of the big vendors are also looking at optical technologies.
Hopefully, we will get an update at a future Hot Chips.
“It seems like it is focused on building something beyond a rectile limited chip like the NVIDIA H100.”
While I’m sure many chips over the age of 65 become rectile limited, I believe you meant reticle?