The NVIDIA L4 is going to be an ultra-popular GPU for one simple reason: its form factor pedigree. The NVIDIA T4 was a hit when it arrived. It offered the company’s tensor cores and solid memory capacity. The real reason for the T4’s success was the form factor. The NVIDIA T4 was a low-profile PCIe card that did not require an additional power cable. That meant that it was extremely easy to integrate into most servers. With this generation, the NVIDIA L4 looks to follow that pedigree for success.
NVIDIA L4 Released 4x NVIDIA T4 Performance in a Similar Form Factor
The NVIDIA L4 is a data center GPU from NVIDIA, but it is far from the company’s fastest. Here are the key specs of the new GPU:
|TF32 Tensor Core||120 teraFLOPS*|
|FP16 Tensor Core||242 teraFLOPS*|
|BFLOAT16 Tensor Core||242 teraFLOPS*|
|FP8 Tensor Core||485 teraFLOPs*|
|INT8 Tensor Core||485 TOPs*|
|GPU memory bandwidth||300GB/s|
|NVENC | NVDEC | JPEG decoders||2 | 4 | 4|
|Max thermal design power (TDP)||72W|
|Form factor||1-slot low-profile, PCIe|
|Interconnect||PCIe Gen4 x16 64GB/s|
The 72W is very important since that allows the card to be powered by the PCIe Gen4 x16 slot without another power cable. Finding auxiliary power in servers that support it is easy, but what made the T4 successful was not requiring that support.
NVIDIA is also pushing on the T4’s NVENC/ NVDEC capabilities. NVIDIA is updating this generation to support features like AV1 per the keynote. That will be important in future workflows when data is uploaded, decoded, an AI transformation applied, and then re-encoded in the future. One of the other main benefits of the NVIDIA T4 was that it was a relatively easy way to add video encode/ decode acceleration in a server.
While this may not be the biggest headline GPU for NVIDIA as it uses a fraction of the power and has a fraction of the performance of something like a NVIDIA H100, it is still an important announcement for the company as this is NVIDIA’s go-anywhere GPU offering.