NVIDIA Tesla T4 AI Inferencing GPU Benchmarks and Review

NVIDIA Tesla T4 Specifications

Here are the key specifications for the Tesla T4:

Overall the NVIDIA Tesla T4 is fairly close in specifications to the NVIDIA GeForce RTX 2060 SUPER. The Tesla T4 has twice the installed memory at 16GB and this is ECC memory. At the same time, the 2060 Super uses more power with higher clock speeds.

Testing the NVIDIA Tesla T4

Here are the key specs of our test setup:

• Motherboard: ASUS WS C621E SAGE Motherboard
• CPU: 2x Intel Xeon Gold 6134 (8 core / 16 Threads)
• GPU: NVIDIA GeForce RTX 2080 SUPER & NVIDIA Tesla T4
• Cooling: Noctua NH-U14S DX-3647 LGA3647
• RAM: 12x MICRON 16GB Low Profile
• SSD: Samsung PM961 1TB
• OS: Windows 10 Pro Workstation / Ubuntu 18.04 LTS

As a quick note here, we added additional cooling to ensure the GPU would not throttle during our tests. We probably had slightly better cooling than most 1U servers.

Here is the obligatory GPU-Z shot of the NVIDIA Tesla T4:

GPU-Z shows the primary stats of our testing the NVIDIA Tesla T4. The GPU clocks in at 585 MHz and can boost up to 1,590 MHz. Pixel Fillrates run at 101.8 GPixels/s, and Texture Fillrate comes in at 254.4 GTexel/s, while memory runs at 1,250 MHz.

This is very important. The low base clocks mean that this card requires solid chassis cooling to perform well by hitting higher boost speeds. We specifically added airflow to ensure that we would not be thermally throttled, but if you deploy this card in a chassis with less airflow, you may see lower results.

Let us move on and start our testing with computing-related benchmarks.