While training has been the big push for the past several years, AI inferencing is seen as the next application that NVIDIA EGX will target. After a model is trained, NVIDIA EGX platforms are designed to take models and do high-speed and power efficient AI inferencing at the edge. During Computex 2019, the company announced the EGX platform.
NVIDIA EGX Platform for Edge AI Inference
The NVIDIA EGX platform is designed for the emerging use case of deploying AI inferencing models to the edge.
Intel has been saying that all one needs is 2nd Gen Intel Xeon Scalable CPUs and their VNNI extensions for edge AI inferencing. NVIDIA, predictably, says that its GPUs from the Jetson Nano to the Tesla T4 is the perfect way to address this market.
The NVIDIA EGX platform will bring the hardware from Jetson platforms to servers with NVIDIA Tesla T4 GPUs along with Mellanox networking and their accompanying software stacks to provide an integrated solution. We were slightly surprised to see Red Hat Openshift being used instead of Ubuntu alternatives given how much more prevalent Ubuntu is in deep learning/ AI deployments.
NVIDIA provided a smart city use case for the EGX platform. It shows how AI inferencing helps to deliver new capabilities to those who need to, for example, monitor city telemetry and video data.
As part of the NVIDIA EGX launch, the company is announcing support with fourteen vendors and even more ISVs.
One example of what we expect to see we covered in the QCT QuantaMicro X11C-8N 2U 8x Intel Xeon E-2100 Microserver with the GPU sled option.
There, QCT months ago showed its microservers with NVIDIA Tesla T4 GPUs.
Final Words
The NVIDIA EGX platform seems an extension of the NVIDIA RTX Server and RTX Server Pod Announced at GTC 2019 which are to bring the Tesla T4 everywhere. NVIDIA has not yet released a Tesla V100 successor and is thus pushing the Tesla T4 to any application that it can find. With the NVIDIA EGX platform, the company is going beyond just hardware and is packaging software and management, but we want to see what else the company will provide with this package.
