NVIDIA, renowned for its advancements in artificial intelligence (AI), has introduced its latest CPU+GPU chip, Grace Hopper, which promises to usher in the next era of AI models and chatbots.
While traditionally known for their role in accelerating graphics rendering for computer games, graphics processing units (GPUs) have demonstrated significantly higher computing power compared to central processing unit (CPU) chips. This led tech companies to adopt GPUs for training AI models due to their ability to perform multiple calculations simultaneously, in parallel.
In 2020, NVIDIA introduced the A100 GPU chip, which proved instrumental in training early iterations of conversational chatbots and image generators. However, within just a short span, the highly advanced H100 Hopper chips have emerged as essential components in data centers that power popular chatbots like ChatGPT. Now, NVIDIA has unveiled a groundbreaking chip that integrates both CPU and GPU capabilities.
The Grace Hopper chip represents a significant leap forward, combining the strengths of CPU and GPU technologies to enhance AI model training and performance. Its introduction marks a new milestone in the ongoing development of AI hardware, enabling more efficient and powerful computing capabilities for AI-related applications.
As the AI landscape continues to evolve, NVIDIA’s Grace Hopper chip aims to play a pivotal role in driving advancements in AI models and chatbot technologies, propelling the field toward unprecedented possibilities.
What are Grace Hopper chips from Nvidia?
According to a press release, Nvidia has created its new chip by combining its Hopper GPU platform with the Grace CPU platform (both named after Grace Hopper, a pioneer of computer programming). The two chips have been connected using Nvidia’s NVLink chip-to-chip (C2C) interconnect technology.
Dubbed GH200, the super chip has 528 GPU tensor cores which can support 480 GB of CPU RAM and 96 GB of GPU RAM. The GPU memory bandwidth on the GH200 is 4TB per second, which is twice as much as the A100 chips.
The super chip also boasts 900GB/s of the coherent memory interface, which is seven times faster than the latest generation PCIe, which has only become available this year. Along with running all Nvidia software such as HPC SDK, Nvidia AI, and Omniverse, the GH200 has 30 times higher aggregate memory bandwidth compared to the A100 chips.
What will chips be used for?
Nvidia, well on its way to becoming a trillion-dollar company, expects the GH200 chips to be used for giant-scale AI and high-performance computing (HPC) applications. At this point in time, one can only imagine AI models and chatbots that are faster and more accurate being built with this superior technology.
The company also plans to use them to build a new exaflop supercomputer capable of performing 1018 floating point operations per second (FLOPS). Two hundred fifty-six of the GH200 chips will be put together to function as one large GPU and have 144 TB of shared memory, about 500 times that of the A100.
“Generative AI is rapidly transforming businesses, unlocking new opportunities, and accelerating discovery in healthcare, finance, business services, and many more industries,” said Ian Buck, vice president of accelerated computing at NVIDIA, in a press release. “With Grace Hopper Superchips in full production, manufacturers worldwide will soon provide the accelerated infrastructure enterprises need to build and deploy generative AI applications that leverage their unique proprietary data.”
Global hyperscalers and supercomputing centers in the U.S. and Europe will get access to the GH200-powered systems later this year, the release added.
