NVIDIA has announced that the H200 will be available from global system manufacturers and cloud service providers starting in the second quarter of 2024.
Unveiled during today’s SC23 event in Denver, the H200 is NVIDIA’s latest GPU geared toward generative AI and high performance computing workloads, which, while based on the same leading Hopper architecture and HGX platform, happens to be the very first to offer HBM3e memory.
HBM3e delivers quite the performance improvement, NVIDIA explains:
With HBM3e, the NVIDIA H200 delivers 141GB of memory at 4.8 terabytes per second, nearly double the capacity and 2.4x more bandwidth compared with its predecessor, the NVIDIA A100.
And here’s how the H200 is expected to change the AI game:
The introduction of H200 will lead to further performance leaps, including nearly doubling inference speed on Llama 2, a 70 billion-parameter LLM, compared to the H100. Additional performance leadership and improvements with H200 are expected with future software updates.
NVIDIA goes on to confirm that the H200 will be available in NVIDIA HGX H200 server boards with four- and eight-way configurations, the latter of which appears to provide serious performance:
An eight-way HGX H200 provides over 32 petaflops of FP8 deep learning compute and 1.1TB of aggregate high-bandwidth memory for the highest performance in generative AI and HPC applications.
Users will also find NVIDIA NVLink and NVSwitch high-speed interconnects, alongside compatibility with the hardware and software of HGX H100 systems.
“To create intelligence with generative AI and HPC applications, vast amounts of data must be efficiently processed at high speed using large, fast GPU memory,” said Ian Buck, vice president of hyperscale and HPC at NVIDIA. “With NVIDIA H200, the industry’s leading end-to-end AI supercomputing platform just got faster to solve some of the world’s most important challenges.”