Top
Author
Updated
16 Dec 2024Form Number
LP1944PDF size
19 pages, 223 KBSubscribed to LP1944.
Thank you for your feedback.
Abstract
The NVIDIA H200 Tensor Core GPU supercharges generative AI and high-performance computing (HPC) workloads with game-changing performance and memory capabilities.
This product guide provides essential presales information to understand the NVIDIA H200 GPU and their key features, specifications, and compatibility. This guide is intended for technical specialists, sales specialists, sales engineers, IT architects, and other IT professionals who want to learn more about the GPUs and consider their use in IT solutions.
Change History
Changes in the December 16, 2024 update:
- Removed the vGPU and Omniverse software part numbers as not supported with the H200 GPUs - NVIDIA GPU software section
Introduction
The NVIDIA H200 Tensor Core GPU supercharges generative AI and high-performance computing (HPC) workloads with game-changing performance and memory capabilities. H200 is the newest addition to NVIDIA’s leading AI and high-performance data center GPU portfolio, bringing massive compute to data centers.
The NVIDIA H200 141GB 700W GPU is offered in the ThinkSystem SR680a V3 server, with eight SXM5 form factor GPU modules and NVIDIA® NVLink® Fabric to create an 8-FC (fully connected) NVLink topology per baseboard. The NVIDIA H200 GPU is also offered as a 4-GPU board in the ThinkSystem SD665-N V3 with four SXM5 GPU modules fully connected using NVLink connections.
Leveraging the power of H200 multi-precision Tensor Cores, an eight-way HGX H200 provides over 32 petaFLOPS of FP8 deep learning compute and over 1.1TB of aggregate HBM memory for the highest performance in generative AI and HPC applications.
Figure 1. ThinkSystem NVIDIA HGX H200 141GB 700W 8-GPU Board in the ThinkSystem SR680a V3 server
Did you know?
To maximize compute performance, H200 is the world’s first GPU with HBM3e memory with 4.8TB/s of memory bandwidth, a 1.4X increase over H100. H200 also expands GPU memory capacity nearly doubled to 141GB. The combination of faster and larger HBM memory accelerates performance of computationally intensive generative AI and HPC applications, while meeting the evolving demands of growing model sizes.
Part number information
The following table shows the part numbers for the 8-GPU and 4-GPU boards. The feature codes contain all H200 GPUs in the SXM form factor plus the NVLink high-speed interconnections between the GPUs.
The table also indicates which GPUs include a 5-year subscription to NVIDIA AI Enterprise Software (NVAIE).
* ThinkSystem NVIDIA H200 NVL 141GB PCIe GPU Gen5 Passive GPU, C3V3 includes a 5-year subscription to NVIDIA AI Enterprise Software (NVAIE). See the NVIDIA AI Enterprise Software section.
The NVIDIA H200 GPU is Controlled which means the GPU is not offered in certain markets, as determined by the US Government.
Features
The NVIDIA H200 Tensor Core GPU supercharges generative AI and HPC with game-changing performance and memory capabilities. As the first GPU with HBM3e, H200’s faster, larger memory fuels the acceleration of generative AI and LLMs while advancing scientific computing for HPC workloads.
NVIDIA HGX™ H200, the world’s leading AI computing platform, features the H200 GPU for the fastest performance. An eight-way HGX H200 provides over 32 petaflops of FP8 deep learning compute and 1.1TB of aggregate high-bandwidth memory for the highest performance in generative AI and HPC applications.
Key AI and HPC workload features:
- Unlock Insights With High-Performance LLM Inference
In the ever-evolving landscape of AI, businesses rely on large language models to address a diverse range of inference needs. An AI inference accelerator must deliver the highest throughput at the lowest TCO when deployed at scale for a massive user base. H200 doubles inference performance compared to H100 when handling LLMs such as Llama2 70B.
- Optimize Generative AI Fine-Tuning Performance
Large language models can be customized to specific business case needs with fine-tuning, low-rank adaptation (LoRA), or retrieval-augmented generation (RAG) methods. These methods bridge the gap between general pretrained results and task-specific solutions, making them essential tools for industry and research applications.
NVIDIA H200’s Transformer Engine and fourth-generation Tensor Cores speed up fine-tuning by 5.5X over A100 GPUs. This performance increase allows enterprises and AI practitioners to quickly optimize and deploy generative AI to benefit their business. Compared to fully training foundation models from scratch, fine-tuning offers better energy efficiency and the fastest access to customized solutions needed to grow business.
- Industry-Leading Generative AI Training
The era of generative AI has arrived, and it requires billion-parameter models to take on the paradigm shift in business operations and customer experiences.
NVIDIA H200 GPUs feature the Transformer Engine with FP8 precision, which provides up to 5X faster training over A100 GPUs for large language models such as GPT-3 175B. The combination of fourth-generation NVLink, which offers 900GB/s of GPU-to-GPU interconnect, PCIe Gen5, and NVIDIA Magnum IO™ software, delivers efficient scalability from small enterprise to massive unified computing clusters of GPUs. These infrastructure advances, working in tandem with the NVIDIA AI Enterprise software suite, make the NVIDIA H200 the most powerful end-to-end generative AI and HPC data center platform.
- Supercharged High-Performance Computing
Memory bandwidth is crucial for high-performance computing applications, as it enables faster data transfer and reduces complex processing bottlenecks. For memory-intensive HPC applications like simulations, scientific research, and artificial intelligence, H200’s higher memory bandwidth ensures that data can be accessed and manipulated efficiently, leading to up to a 110X faster time to results.
The NVIDIA data center platform consistently delivers performance gains beyond Moore’s Law. And H200’s breakthrough AI capabilities further amplify the power of HPC+AI to accelerate time to discovery for scientists and researchers working on solving the world’s most important challenges.
- Reduced Energy and TCO
In a world where energy conservation and sustainability are top of mind, the concerns of business leaders and enterprises have evolved. Enter accelerated computing, a leader in energy efficiency and TCO, particularly for workloads that thrive on acceleration, such as HPC and generative AI.
With the introduction of H200, energy efficiency and TCO reach new levels. This cutting-edge technology offers unparalleled performance, all within the same power profile as H100. AI factories and at-scale supercomputing systems that are not only faster but also more eco-friendly deliver an economic edge that propels the AI and scientific community forward. For at-scale deployments, H200 systems provide 5X more energy savings and 4X better cost of ownership savings over the NVIDIA Ampere architecture generation.
Technical specifications
The following table lists the NVIDIA H200 GPU specifications.
* Without / with structural sparsity enabled
Server support
The following tables list the ThinkSystem servers that are compatible.
- Contains 8 separate GPUs connected via high-speed interconnects
Operating system support
Operating system support is based on that of the supported servers. See the SR680a V3 server product guide for details: https://lenovopress.lenovo.com/lp1909-thinksystem-sr680a-v3-server
NVIDIA GPU software
This section lists the NVIDIA software that is available from Lenovo.
NVIDIA AI Enterprise Software
Lenovo offers the NVIDIA AI Enterprise (NVAIE) cloud-native enterprise software. NVIDIA AI Enterprise is an end-to-end, cloud-native suite of AI and data analytics software, optimized, certified, and supported by NVIDIA to run on VMware vSphere and bare-metal with NVIDIA-Certified Systems™. It includes key enabling technologies from NVIDIA for rapid deployment, management, and scaling of AI workloads in the modern hybrid cloud.
NVIDIA AI Enterprise is licensed on a per-GPU basis. NVIDIA AI Enterprise products can be purchased as either a perpetual license with support services, or as an annual or multi-year subscription.
- The perpetual license provides the right to use the NVIDIA AI Enterprise software indefinitely, with no expiration. NVIDIA AI Enterprise with perpetual licenses must be purchased in conjunction with one-year, three-year, or five-year support services. A one-year support service is also available for renewals.
- The subscription offerings are an affordable option to allow IT departments to better manage the flexibility of license volumes. NVIDIA AI Enterprise software products with subscription includes support services for the duration of the software’s subscription license
The features of NVIDIA AI Enterprise Software are listed in the following table.
Note: Maximum 10 concurrent VMs per product license
The following table lists the ordering part numbers and feature codes.
Find more information in the NVIDIA AI Enterprise Sizing Guide.
NVIDIA HPC Compiler Software
Regulatory approvals
The NVIDIA H200 GPU has the following regulatory approvals:
- RCM
- BSMI
- CE
- FCC
- ICES
- KCC
- cUL, UL
- VCCI
Seller training courses
The following sales training courses are offered for employees and partners (login required). Courses are listed in date order.
-
Partner Technical Webinar - NVIDIA Portfolio
2024-11-06 | 60 minutes | Employees and Partners
DetailsPartner Technical Webinar - NVIDIA Portfolioin this 60-minute replay, Jason Knudsen of NVIDIA presented the NVIDIA Computing Platform. Jason talked about the full portfolio from GPUs to Networking to AI Enterprise and NIMs.
Published: 2024-11-06
Length: 60 minutesCourse code: 110124Start the training:
Employee link: Grow@Lenovo
Partner link: Lenovo Partner Learning -
NVIDIA Data Center GPU Portfolio
2024-09-26 | 11 minutes | Employees and Partners
DetailsNVIDIA Data Center GPU PortfolioThis course equips Lenovo and partner technical sellers with the knowledge to effectively communicate the positioning of NVIDIA's data center GPU portfolio, enhancing your ability to showcase its key advantages to clients.
Published: 2024-09-26
Upon completion of this training, you will be familiar with the following:
• Data Center GPUs for AI and HPC
• Data Center GPUs for Graphics
• GPU comparisons
Length: 11 minutesCourse code: DAINVD201Start the training:
Employee link: Grow@Lenovo
Partner link: Lenovo Partner Learning -
Q2 Solutions Launch TruScale GPU Next Generation Management in the AI Era Quick Hit
2024-09-10 | 6 minutes | Employees and Partners
DetailsQ2 Solutions Launch TruScale GPU Next Generation Management in the AI Era Quick HitThis Quick Hit focuses on Lenovo announcing additional ways to help you build, scale, and evolve your customer’s private AI faster for improved ROI with TruScale GPU as a Service, AI-driven systems management, and infrastructure transformation services.
Published: 2024-09-10
Length: 6 minutesCourse code: SXXW2543aStart the training:
Employee link: Grow@Lenovo
Partner link: Lenovo Partner Learning -
VTT AI: The NetApp AIPod with Lenovo for NVIDIA OVX
2024-08-13 | 38 minutes | Employees and Partners
DetailsVTT AI: The NetApp AIPod with Lenovo for NVIDIA OVXAI, for some organizations, is out of reach, due to cost, integration complexity, and time to deployment. Previously, organizations relied on frequently retraining their LLMs with the latest data, a costly and time-consuming process. The NetApp AIPod with Lenovo for NVIDIA OVX combines NVIDIA-Certified OVX Lenovo ThinkSystem SR675 V3 servers with validated NetApp storage to create a converged infrastructure specifically designed for AI workloads. Using this solution, customers will be able to conduct AI RAG and inferencing operations for use cases like chatbots, knowledge management, and object recognition.
Published: 2024-08-13
Topics covered in this VTT session include:
• Where Lenovo fits in the solution
• NetApp AIPod with Lenovo for NVIDIA OVX Solution Overview
• Challenges/pain points that this solution solves for enterprises deploying AI
• Solution value/benefits of the combined NetApp, Lenovo, and NVIDIA OVX-Certified Solution
Length: 38 minutesCourse code: DVAI206Start the training:
Employee link: Grow@Lenovo
Partner link: Lenovo Partner Learning -
Introduction to Artificial Intelligence
2024-08-02 | 11 minutes | Employees and Partners
DetailsIntroduction to Artificial IntelligenceIMPORTANT: If you receive the following error message:
Published: 2024-08-02
"There is an issue with this slide content. Please contact your administrator”, please change your VPN location setting and try again. We are actively working on fixing this issue. Thank you for your understanding!
This NVIDIA course aims to answer questions such as:
• What is AI?
• Why are enterprises so interested in it?
• How does AI happen?
• Why are GPUs so important for it?
• What does a good AI solution look like?
Course Objectives:
By the end of this training, you should be able to:
1. Describe AI on a high level and list a few common enterprise use cases
2. List how enterprises benefit from AI
3. Distinguish between Training and Inference
4. Say how GPUs address known bottlenecks in a typical AI pipeline
5. Tell a customer why NVIDIA’s AI solutions are well-respected in the market
Length: 11 minutesCourse code: DAINVD104r2Start the training:
Employee link: Grow@Lenovo
Partner link: Lenovo Partner Learning -
GPU Fundamentals
2024-08-02 | 10 minutes | Employees and Partners
DetailsGPU FundamentalsIMPORTANT: If you receive the following error message:
Published: 2024-08-02
"There is an issue with this slide content. Please contact your administrator”,
please change your VPN location setting and try again. We are actively working on fixing this issue. Thank you for your understanding.
This NVIDIA course introduces you to two devices that a computer typically uses to process information – the CPU and the GPU. We’ll discuss their differences and look at how the GPU overcomes the limitations of the CPU. We will also talk about the value GPUs bring to modern-day enterprise computing.
Course Objectives:
By the end of this training, you should be able to:
1. Distinguish between serial and parallel processing
2. Explain what a GPU is and what it does at a high level
3. Articulate the value of GPU computing for enterprises
4. List three typical GPU-accelerated workloads and a few uses cases
5. Recommend the appropriate NVIDIA GPU for its corresponding enterprise computing workloads
Length: 10 minutesCourse code: DAINVD103r2Start the training:
Employee link: Grow@Lenovo
Partner link: Lenovo Partner Learning -
Key NVIDIA Use Cases for Industry Verticals
2024-08-02 | 32 minutes | Employees and Partners
DetailsKey NVIDIA Use Cases for Industry VerticalsIMPORTANT: If you receive the following error message:
Published: 2024-08-02
"There is an issue with this slide content. Please contact your administrator”,
please change your VPN location setting and try again. We are actively working on fixing this issue. Thank you for your understanding.
In this NVIDIA course, you will learn about key AI use cases driving innovation and change across Automotive, Financial Services, Energy, Healthcare, Higher Education, Manufacturing, Retail and Telco industries.
Course Objectives:
By the end of this training, you should be able to:
1. Discuss common AI use cases across a broad range of industry verticals
2. Explain how NVIDIA’s AI software stack speeds up time to production for AI projects in multiple industry verticals
Length: 32 minutesCourse code: DAINVD108Start the training:
Employee link: Grow@Lenovo
Partner link: Lenovo Partner Learning -
Generative AI Overview
2024-08-02 | 17 minutes | Employees and Partners
DetailsGenerative AI OverviewIMPORTANT: If you receive the following error message:
Published: 2024-08-02
"There is an issue with this slide content. Please contact your administrator”, please change your VPN location setting and try again. We are actively working on fixing this issue. Thank you for your understanding!
Since ChatGPTs debut in November of 2022, it has become clear that Generative AI has the potential to revolutionize many aspects of our personal and professional lives. This NVIDIA course aims to answer questions such as:
• What are the Generative AI market trends?
• What is generative AI and how does it work?
Course Objectives:
By the end of this training, you should be able to:
1. Discuss the Generative AI market trends and the challenges in this space with your customers.
2. Explain what Generative AI is and how the technology works to help enterprises to unlock new opportunities for the business.
3. Present a high-level overview of the steps involved in building a Generative AI application.
Length: 17 minutesCourse code: DAINVD106r2Start the training:
Employee link: Grow@Lenovo
Partner link: Lenovo Partner Learning -
Retrieval Augmented Generation
2024-08-02 | 15 minutes | Employees and Partners
DetailsRetrieval Augmented GenerationIMPORTANT: If you receive the following error message:
Published: 2024-08-02
"There is an issue with this slide content. Please contact your administrator”, please change your VPN location setting and try again. We are actively working on fixing this issue. Thank you for your understanding!
In this NVIDIA course, Dave Barry, Senior Solutions Architect, talks about a technique known as Retrieval Augmented Generation (RAG). It is a powerful tool for enhancing the accuracy and reliability of Generative AI models with facts fetched from external sources.
This course requires prior knowledge of Generative AI concepts, such as the difference between model training and inference. Please refer to relevant courses within this curriculum.
Course Objectives:
By the end of this training, you should be able to:
1. Explain the limitations of large language models to customers
2. Articulate the value of RAG to enterprises
3. Demo an NVIDIA RAG workflow with a video
4. Drive TCO conversations using an authentic use case
Length: 15 minutesCourse code: DAINVD107Start the training:
Employee link: Grow@Lenovo
Partner link: Lenovo Partner Learning -
AI Industry Use Cases & Solutions
2024-08-02 | 25 minutes | Employees and Partners
DetailsAI Industry Use Cases & SolutionsIMPORTANT: If you receive the following error message:
Published: 2024-08-02
"There is an issue with this slide content. Please contact your administrator”, please change your VPN location setting and try again. We are actively working on fixing this issue. Thank you for your understanding!
This NVIDIA course aims to answer the question:
• How does NVIDIA bring AI solutions to market with and through the partner ecosystem?
Course Objectives:
By the end of this training, you should be able to:
1. Think of solutions in terms of an industry and use case approach
2. Develop solutions that address the industry-specific challenges (with FSI as the illustrative model)
3. Engage customers with their conversations and advance deals with stakeholder’s concerns in mind
4. Replicate NVIDIA’s best practices and ecosystem engagement strategies appropriately
Length: 25 minutesCourse code: DAINVD105r2Start the training:
Employee link: Grow@Lenovo
Partner link: Lenovo Partner Learning -
Partner Technical Webinar - NVIDIA Smart Spaces
2024-07-24 | 60 minutes | Employees and Partners
DetailsPartner Technical Webinar - NVIDIA Smart SpacesIn this 60-minute replay, Alex Pazos, NVIDIA BDM for Smart Spaces, reviewed the NVIDIA AI for Smart Spaces framework and use cases. Alex reviewed the Metropolus Framework and the Smart Spaces ecosystem. Then he reviewed several use cases including sports stadiums, warehouses, airports, and roadways.
Published: 2024-07-24
Length: 60 minutesCourse code: 071924Start the training:
Employee link: Grow@Lenovo
Partner link: Lenovo Partner Learning -
Guidance for Selling NVIDIA Products at Lenovo for ISG
2024-07-01 | 25 minutes | Employees and Partners
DetailsGuidance for Selling NVIDIA Products at Lenovo for ISGThis course gives key talking points about the Lenovo and NVIDIA partnership in the Data Center. Details are included on where to find the products that are included in the partnership and what to do if NVIDIA products are needed that are not included in the partnership. Contact information is included if help is needed in choosing which product is best for your customer. At the end of this session sellers should be able to explain the Lenovo and NVIDIA partnership, describe the products Lenovo can sell through the partnership with NVIDIA, help a customer purchase other NVIDIA product, and get assistance with choosing NVIDIA products to fit customer needs.
Published: 2024-07-01
Length: 25 minutesCourse code: DNVIS102Start the training:
Employee link: Grow@Lenovo
Partner link: Lenovo Partner Learning -
Think AI Weekly: Lenovo AI PCs & AI Workstations
2024-05-23 | 60 minutes | Employees Only
DetailsThink AI Weekly: Lenovo AI PCs & AI WorkstationsJoin Mike Leach, Sr. Manager, Workstations Solutions and Pooja Sathe, Director Commercial AI PCs as they discuss why Lenovo AI Developer Workstations and AI PCs are the most powerful, where they fit into the device to cloud ecosystem, and this week’s Microsoft announcement, Copilot+PC
Published: 2024-05-23
Length: 60 minutesCourse code: DTAIW105Start the training:
Employee link: Grow@Lenovo -
VTT Cloud Architecture: NVIDIA Using Cloud for GPUs and AI
2024-05-22 | 60 minutes | Employees Only
DetailsVTT Cloud Architecture: NVIDIA Using Cloud for GPUs and AIJoin JD Dupont, NVIDIA Head of Americas Sales, Lenovo partnership and Veer Mehta, NVIDIA Solution Architect on an interactive discussion about cloud to edge, designing cloud Solutions with NVIDIA GPUs and minimizing private\hybrid cloud OPEX with GPUs. Discover how you can use what is done at big public cloud providers for your customers. We will also walk through use cases and see a demo you can use to help your customers.
Published: 2024-05-22
Length: 60 minutesCourse code: DVCLD212Start the training:
Employee link: Grow@Lenovo -
Partner Technical Webinar - NVidia
2023-12-11 | 60 minutes | Employees and Partners
DetailsPartner Technical Webinar - NVidiaIn this 60-minute replay, Brad Davidson of Nvidia will help us recognize AI Trends, and Discuss Industry Verticals Marketing.
Published: 2023-12-11
Length: 60 minutesCourse code: 120823Start the training:
Employee link: Grow@Lenovo
Partner link: Lenovo Partner Learning
Related links
For more information, refer to these documents:
- ThinkSystem and ThinkAgile GPU Summary:
https://lenovopress.lenovo.com/lp0768-thinksystem-thinkagile-gpu-summary - ServerProven compatibility:
https://serverproven.lenovo.com/ - NVIDIA H200 product page:
https://www.nvidia.com/en-us/data-center/h200/ - NVIDIA Hopper Architecture page
https://www.nvidia.com/en-us/data-center/technologies/hopper-architecture/ - ThinkSystem SR680a V3 product guide
https://lenovopress.lenovo.com/lp1909-thinksystem-sr680a-v3-server
Trademarks
Lenovo and the Lenovo logo are trademarks or registered trademarks of Lenovo in the United States, other countries, or both. A current list of Lenovo trademarks is available on the Web at https://www.lenovo.com/us/en/legal/copytrade/.
The following terms are trademarks of Lenovo in the United States, other countries, or both:
Lenovo®
ServerProven®
ThinkAgile®
ThinkSystem®
The following terms are trademarks of other companies:
AMD is a trademark of Advanced Micro Devices, Inc.
Intel® is a trademark of Intel Corporation or its subsidiaries.
Linux® is the trademark of Linus Torvalds in the U.S. and other countries.
Windows® is a trademark of Microsoft Corporation in the United States, other countries, or both.
Other company, product, or service names may be trademarks or service marks of others.
Configure and Buy
Please select a locale
Full Change History
Changes in the December 16, 2024 update:
- Removed the vGPU and Omniverse software part numbers as not supported with the H200 GPUs - NVIDIA GPU software section
Changes in the December 11, 2024 update:
- ThinkSystem NVIDIA H200 NVL 141GB PCIe GPU Gen5 Passive GPU, C3V3 includes a 5-year subscription to NVIDIA AI Enterprise Software (NVAIE) - Part number information section
Changes in the November 14, 2024 update:
- Added the following DW PCIe adapter:
- ThinkSystem NVIDIA H200 NVL 141GB PCIe GPU Gen5 Passive GPU, C3V3
Changes in the October 10, 2024 update:
- Added the following 4-GPU board:
- ThinkSystem NVIDIA HGX H200 141GB 700W 4-GPU Board, C3V2
Changes in the September 15, 2023 update:
- Added the Controlled status column to Table 1 - Part number information section
First published: April 23, 2024
Course Detail
Employees Only Content
The content in this document with a is only visible to employees who are logged in. Logon using your Lenovo ITcode and password via Lenovo single-signon (SSO).
The author of the document has determined that this content is classified as Lenovo Internal and should not be normally be made available to people who are not employees or contractors. This includes partners, customers, and competitors. The reasons may vary and you should reach out to the authors of the document for clarification, if needed. Be cautious about sharing this content with others as it may contain sensitive information.
Any visitor to the Lenovo Press web site who is not logged on will not be able to see this employee-only content. This content is excluded from search engine indexes and will not appear in any search results.
For all users, including logged-in employees, this employee-only content does not appear in the PDF version of this document.
This functionality is cookie based. The web site will normally remember your login state between browser sessions, however, if you clear cookies at the end of a session or work in an Incognito/Private browser window, then you will need to log in each time.
If you have any questions about this feature of the Lenovo Press web, please email David Watts at dwatts@lenovo.com.
