NVIDIA Tesla V100 Volta Update at Hot Chips 2017

0
NVIDIA Volta Hot Chips 2017 Cover
NVIDIA Volta Hot Chips 2017 Cover

At Hot Chips 2017, NVIDIA presented more information about their new Volta architecture. Specifically, the topic was the NVIDIA Tesla V100 GPU. We previously covered the NVIDIA Tesla V100 at GTC 2017. In the meantime, NVIDIA has started to ship the Volta based GPUs in limited quantities and this quarter we expect the DGX-1 to get its V100 update. In the deep learning space, this is the generation where NVIDIA hardware is clearly leap frogging AMD’s newest offering.

NVIDIA Tesla V100 Overview
NVIDIA Tesla V100 Overview

One can see here that the NVIDIA Tesla V100 has 80 SMs active. The small footnote on the slide we could see clearly from our seat. It says that a full GP100 chip contains 84 SMs so NVIDIA is binning products to enable higher yields.

In terms of performance, the NVIDIA Tesla V100 numbers are sometimes staggering for a generational improvement.

NVIDIA V100 And P100 Comparison
NVIDIA Tesla V100 And P100 Comparison

The key here is that NVIDIA is moving to larger caches, and adding specific accelerators for the deep learning space while beefing up overall bandwidth.

We did want to take a second and show the depiction of the SM core. One will notice that the SM core has large Tensor Cores. It is clear what NVIDIA’s focus is for this generation.

NVIDIA V100 SM Core
NVIDIA V100 SM Core

From the illustration, this is a ton of die space dedicated to Tensor Cores. One has to wonder whether NVIDIA will make the decision to bifurcate its deep learning chips from its consumer/ video chips with this generation. It becomes hard to fathom how costly this would be on a gaming GPU where it would get little to no use.

We are not going to spend much time regurgitating the text NVIDIA presented on the SM side. Like most architectural advancements, the new NVIDIA Volta GV100 SM is designed for more performance.

NVIDIA Volta GV100 SM
NVIDIA Volta GV100 SM

Likewise, here is the microarchitecture view:

NVIDIA Volta V100 SM Microarchitecture
NVIDIA Volta V100 SM Microarchitecture

And the sub-core view:

NVIDIA Volta V100 Sub Core
NVIDIA Volta V100 Sub Core

Key here are the two 4x4x4 Tensor Cores. That is the secret sauce that is making Volta a revolutionary leap over an evolutionary one.

One other area NVIDIA spent time on was the L1 cache and shared memory.

NVIDIA Volta V100 Shared Memory
NVIDIA Volta V100 Shared Memory

One can now dedicate cache as shared memory if you know your workload can take advantage of such data locality. As NVIDIA beefs up L1 and L2 caches, we are seeing a much more compute driven architecture than something required for gaming.

NVIDIA V100 Tensor Core
NVIDIA V100 Tensor Core

The Tensor Core is the allows one to do mixed precision matrix math using 4×4 matrices.

NVIDIA V100 Tensor Core Math
NVIDIA V100 Tensor Core Math

This is a key operation when training deep learning models. With improved scheduling, NVIDIA can do 16×16 matrix math.

NVIDIA V100 Tensor Synchronization
NVIDIA V100 Tensor Synchronization

Here is another view NVIDIA presented on the Volta tensor operation showing how one can use lower precision FP16 inputs and arrive at a FP32 output.

NVIDIA V100 Tensor Operation
NVIDIA V100 Tensor Operation

In terms of impact, NVIDIA claims that some operations will see a 9.3x speedup.

NVIDIA V100 And P100 Deep Learning Benefit
NVIDIA V100 And P100 Deep Learning Benefit

We do want to note here that NVIDIA is combining CUDA 8 to CUDA 9 software gains along with hardware gains in a specific workload.

NVLINK Updates

NVIDIA is claiming that the new GV100 NVLINK will offer even more bandwidth of up to 1.9x what we saw with the GP100.

NVIDIA NV Link Nodes
NVIDIA NV Link Nodes

The GV100 will have up to 6 NVLINK connections which allow for a number of topologies to facilitate GPU to GPU connections. This is certainly a much higher-end way to get more GPU to GPU bandwidth than we have seen in servers featuring PCIe switches. Single Root v Dual Root for Deep Learning GPU to GPU Systems.

Final Words

There are a few key takeaways from the NVIDIA Tesla Volta V100 presentation. First, it is clear that driving data center sales is an architectural goal of this generation of products. Second, one does have to ponder whether all of this added logic makes sense in consumer parts. One of the keys to NVIDIA’s success in the deep learning space has been that high-performance CUDA training could happen on the desktop using the company’s gaming GPUs. From the diagrams, NVIDIA is showing, and the logic portions, one has to wonder whether the Tensor Core will be part of the gaming line in any form. While we would expect the NVIDIA Tesla V100 to have premium features at the price, there appears to be a lot of silicon in the V100 design dedicated to non-traditional GPU (gaming) tasks.

SHARE
Previous articleAMD Vega10 Update from Hot Chips 2017
Next articleCase Study on the Google TPU and GDDR5 from Hot Chips 29
Patrick has been running STH since 2009 and covers a wide variety of SME, SMB, and SOHO IT topics. Patrick is a consultant in the technology industry and has worked with numerous large hardware and storage vendors in the Silicon Valley. The goal of STH is simply to help users find some information about server, storage and networking, building blocks. If you have any helpful information please feel free to post on the forums.

LEAVE A REPLY

Please enter your comment!
Please enter your name here