NVIDIA has unveiled a groundbreaking advancement in the realm of AI supercomputing with its latest generation of chips. This new technology is not just an upgrade in terms of capability; it’s specifically tailored to enhance the performance of AI applications and Large Language Models (LLMs) such as OpenAI’s GPT-4. These chips are expected to significantly influence the future of deep learning and a variety of other complex tasks.
Central to this new lineup is the HGX H200 GPU, based on NVIDIA’s innovative “Hopper” architecture. This new chip is a successor to the popular H100 GPU and marks NVIDIA’s first foray into utilizing HBM3e memory. This upgrade results in faster performance and greater memory capacity, making it particularly efficient for handling large language models. The H200 boasts an impressive 141GB of memory and a 4.8 terabytes per second bandwidth, a substantial improvement over its predecessor, the NVIDIA A100.
From an AI perspective, the HGX H200 is a game changer. It doubles the inference speed on Llama 2, an LLM with 70 billion parameters, compared to the H100. Its versatility is evident in its compatibility with various data center types, including on-premises, cloud, hybrid-cloud, and edge systems. Major cloud providers like Amazon Web Services, Google Cloud, Microsoft Azure, and Oracle Cloud Infrastructure are set to deploy this technology. NVIDIA anticipates the chip’s release in the second quarter of 2024.
Another significant product from NVIDIA is the GH200 Grace Hopper “superchip.” This innovative chip combines the HGX H200 GPU with the Arm-based NVIDIA Grace CPU through NVIDIA’s NVLink-C2C interlink. Designed specifically for supercomputers, this superchip aims to revolutionize the way scientists and researchers address complex AI and high-performance computing (HPC) applications, especially those involving large data sets.
The GH200 is slated to be a core component in over 40 AI supercomputers around the world, with involvement from companies like Dell, Eviden, Hewlett Packard Enterprise (HPE), Lenovo, QCT, and Supermicro. A notable example is HPE’s Cray EX2500 supercomputers, which will incorporate quad GH200s, allowing for scalability up to tens of thousands of Grace Hopper Superchip nodes.
A particularly exciting implementation of this technology will be seen in JUPITER, a supercomputer at the Jülich facility in Germany. Poised to be the “world’s most powerful AI system” upon its installation in 2024, JUPITER features a liquid-cooled architecture that integrates nearly 24,000 NVIDIA GH200 Superchips. This supercomputer is set to contribute significantly to scientific advancements in areas like climate and weather prediction, drug discovery, quantum computing, and industrial engineering.
JUPITER will utilize NVIDIA’s custom software solutions, which, while easing development, also foster a dependence on NVIDIA hardware for supercomputing groups. These advancements come at a time when NVIDIA is increasingly focusing on AI and data center sectors, which now constitute a major portion of its revenue. The company recently reported a record $10.32 billion revenue in this segment, a substantial increase from the previous year, signaling the growing importance of these technologies in NVIDIA’s portfolio.
With the introduction of these new chips, NVIDIA is poised to extend its dominance in the AI supercomputing sector. The company’s recent achievement in breaking its own AI training benchmark record using the older H100 technology is a testament to its ongoing commitment to innovation and leadership in this field. The new GPU and superchip are not just incremental updates; they represent a significant stride forward in AI and supercomputing technology, promising to unlock new potentials in various research and industry domains.
