Is NVIDIA more of a hardware or software company? Probably the latter, according to Manuvir Das, Head of Enterprise Computing at NVIDIA, who spoke at MIT Technology Review’s Future Compute event yesterday and mentioned that “at NVIDIA, we spend 20% of our time on the hardware and 80% of our time on the software.” Das’ statement came on the same day that NVIDIA began shipping the DGX H100, a new system for powering AI applications that features eight NVIDIA H100 GPUs with 640 GB of GPU memory. As noted by John Carmack, NVIDIA’s H100 GPUs are selling for as high as $45,000 on eBay.
From an NVIDIA Newsroom post:
Each NVIDIA H100 Tensor Core GPU in a DGX H100 system provides on average about 6x more performance than prior GPUs. A DGX H100 packs eight of them, each with a Transformer Engine designed to accelerate generative AI models.
The eight H100 GPUs connect over NVIDIA NVLink to create one giant GPU. Scaling doesn’t stop there: organizations can connect hundreds of DGX H100 nodes into an AI supercomputer using the 400 Gbps ultra-low latency NVIDIA Quantum InfiniBand, twice the speed of prior networks.
"At NVIDIA, we spend 20% of our time on the hardware and 80% of our time on the software.
— Simon Erickson (@7Innovator) May 1, 2023
We are continually working on the algorithms, to understand what our hardware can and cannot do. AI is a full-stack problem."
– Manuvir Das, Head of Enterprise Computing$NVDA… pic.twitter.com/zFM4XNbjVJ