NVIDIA had one of the more interesting updates this week at SIGGRAPH 2023. The NVIDIA L40S is a 350W PCIe card GPU with more performance than the NVIDIA L40. In case you were wondering, since we only received a stock press image for this GPU, we decided to use Adobe Photoshop Beta for its generative fill for the images here.
NVIDIA L40S GPU for Data Center Visualization Launched
The new NVIDIA L40S has many of the same specs as the previous NVIDIA L40. The NVIDIA L40 is designed to be packed into PCIe servers where chassis airflow cools the GPUs. We have reviewed many of these over the years with cards like the NVIDIA A100, but cards like the NVIDIA A40 48GB GPU have been popular as well. The NVIDIA L40 and L40S are the successors to the A40 in many ways.
If you saw our cover image for the NVIDIA A40 review, you will see a good example of a deployment for the L40 and L40S.
Here are the key specs of the new card:
|Architecture||NVIDIA Ada Lovelace Architecture|
|Process Size||4nm NVIDIA Custom Process|
|Die Size||608.44 mm|
|CUDA Parallel Processing Cores||18,176|
|NVIDIA Tensor Cores (4th Gen)||568|
|NVIDIA RT Cores (3rd Gen)||142|
|Peak FP32 TFLOPS (non-Tensor)||91.6|
|Peak FP16 Tensor TFLOPS with FP16 Accumulate||366.5 | 733*|
|Peak FP8 Tensor TFLOPs with FP16 Accumulate||733 | 1466*|
|Peak TF32 Tensor TFLOPS||183 | 366*|
|Peak BF16 Tensor TFLOPS with FP32 Accumulate||366.5 | 733*|
|Peak INT8 Tensor TOPS||733 | 1466*|
|Peak INT4 Tensor TOPS||733 | 1466*|
|RT Core Performance TFLOPS||212|
|GPU Memory||48 GB GDDR6 with ECC|
|Memory Bandwidth||864 GB/s|
|Interconnect||x16 PCIe Gen4 (no NVLink)|
|Max Power Consumption||350W|
|Graphics Bus||PCI Express 4.0 x16|
|Display Connectors||DP 1.4a
Supports NVIDIA Mosaic and Quadro Sync
|Display Max Resolution / Quantity||Up to four 5K Monitors @ 60Hz per card or dual 8K displays @ 60Hz
Each display port can support 4K @ 120hz with 30-bit color
|Form Factor||4.4” H x 10.5” L – Dual Slot|
The biggest difference seems to be the 350W power, up from 300W on the L40. Adding more power increases the average clock speed the GPU can run at and therefore increases performance.
The L40S is interesting because it was not long ago where a 300W GPU in a PCIe card slot was considered a practical limit. Using GDDR6 instead of HBM lowers the cost of the cards and therefore can make these attractive cards for either remote visualization or even some forms of GPU compute.
Since these are designed to run applications like generative AI for creative professionals, it seemed to make sense to generate a few fills for the background of the card.