Image: NVIDIA

NVIDIA today introduced its A2 Tensor Core GPU, an entry-level GPU that’s capable of delivering 20 percent more inference performance than CPUs despite its low power and small footprint. Inference is a component of the Deep Learning process whereby trained models developed during the training (i.e., fact gathering or learning) phase are actually applied and processed in an application. NVIDIA’s A2 Tensor Core GPU provides up to 1.3x more performance in various intelligent edge use cases for this purpose.

From NVIDIA:

A2’s versatility, compact size, and low power exceed the demands for edge deployments at scale, instantly upgrading existing entry-level CPU servers to handle inference. Servers accelerated with A2 GPUs deliver higher inference performance versus CPUs and more efficient intelligent video analytics (IVA) deployments than previous GPU generations—all at an entry-level price point. […]

AI inference is deployed to make consumer lives more convenient through real-time experiences, and enables them to gain insights on trillions of end-point sensors and cameras. Compared to CPU-only servers, the servers built with NVIDIA A2 Tensor Core GPU offer up to 20X more inference performance, instantly upgrading any server to handle modern AI.

A2 Tensor Core Specifications

Peak FP324.5 TF
TF32 Tensor Core9 TF | 18 TF
BFLOAT16 Tensor Core18 TF | 36 TF
Peak FP16 Tensor Core18 TF | 36 TF
Peak INT8 Tensor Core36 TOPS | 72 TOPS
Peak INT4 Tensor Core72 TOPS | 144 TOPS
RT Cores10
Media engines1 video encoder
2 video decoders (includes AV1 decode)
GPU memory16GB GDDR6
GPU memory bandwidth200GB/s
InterconnectPCIe Gen4 x8
Form factor1-slot, low-profile PCIe
Max thermal design power (TDP)40–60W (configurable)
Virtual GPU (vGPU) software supportNVIDIA Virtual PC (vPC), NVIDIA Virtual Applications (vApps), NVIDIA RTX Virtual Workstation (vWS), NVIDIA AI Enterprise, NVIDIA Virtual Compute Server (vCS)

Source: NVIDIA

Don’t Miss Out on More FPS Review Content!

Our weekly newsletter includes a recap of our reviews and a run down of the most popular tech news that we published.

Leave a comment