Breaking Barriers in AI Inference: Lenovo ThinkSystem Servers Shine in MLPerf v4.1 - Unleashing the Power of AI-Ready Infrastructure

In a world where AI is revolutionizing industries and transforming businesses, Lenovo is proud to announce that the ThinkSystem servers have broken barriers and taken the top spot in an impressive 54 out of 79 MLPerf v4.1 benchmarks! This remarkable achievement solidifies our position as a leader in the AI infrastructure market, empowering organizations to unlock the full potential of their AI initiatives.

With this groundbreaking performance, Lenovo ThinkSystem servers have demonstrated their ability to handle complex AI workloads with ease and efficiency, making them an ideal choice for organizations looking to accelerate their AI projects. Whether you're a leading financial institution, a cutting-edge tech firm, or a forward-thinking healthcare provider, Lenovo ThinkSystem servers are designed to help you stay ahead of the curve in the rapidly evolving world of AI.

Our ThinkSystem SR680a V3 and SR685a V3 systems have taken center stage by competing against each other in several generative AI benchmarks, showcasing the incredible power of Lenovo's server configurations. The results are impressive:

• GPT-99 Champion: The ThinkSystem SR680a V3 (Intel) with 8x NVIDIA H200 Tensor Core SXM GPUs, each with 141GB, took top honors, leveraging its Intel 8568Y 48-core processor and advanced memory architecture to deliver exceptional performance.
• Llama 2 Leaderboard: We secured victory again, this time with the Lenovo ThinkSystem SR685a V3 (AMD) with 8x NVIDIA H200 SXM GPUs with 141GB, highlighting the versatility of Lenovo's server configurations across different architectures.
• Stable Diffusion XL Speedster: Lenovo ThinkSystem SR680a V3 (Intel) with 8x NVIDIA H200 SXM GPUs with 141GB came out on top in this highly competitive category, demonstrating its ability to handle complex AI workloads and scale performance for demanding applications.

With finishes at or near the top of the pack across numerous MLPerf Inference tests, these results showcase the capabilities of Lenovo ThinkSystem servers in various AI inference scenarios. Whether you're developing cutting-edge AI models or processing large datasets, Lenovo's server configurations provide the scalability and performance you need to drive innovation forward.

The following table provides a breakdown of our results.

Our benchmarks resulted in world records against all tested systems showcasing Lenovo’s consistent improvement to achieve best-in-class results for our customers. These key results, in particular, show how powerful our systems are in those specific categories:

• ThinkSystem SR685a V3(8x H200-SXM-141GB, NVIDIA TensorRT) - 1^st place on retinanet offline
• ThinkSystem SR680a V3 (8x H200-SXM-141GB, NVIDIA TensorRT) - 1^st place on bert-99.9 server
• ThinkSystem SR680a V3 (8x H200-SXM-141GB, NVIDIA TensorRT) – 1^st place on gptj-99 offline
• ThinkSystem SR680a V3 (8x H200-SXM-141GB, NVIDIA TensorRT) – 1^st place on gptj-99.9 offline
• ThinkSystem SR680a V3 (8x H200-SXM-141GB, NVIDIA TensorRT) – 1^st place on stable-diffusion-xl server
• ThinkSystem SR680a V3 (8x H200-SXM-141GB, NVIDIA TensorRT) – 2^nd place on stable-diffusion-xl offline
• ThinkSystem SR680a V3 (8x H200-SXM-141GB, NVIDIA TensorRT) - 2^nd place on bert-99.9 server
• ThinkSystem SR680a V3 (8x H200-SXM-141GB, NVIDIA TensorRT) – 3^rd place on gptj-99 server
• ThinkSystem SR680a V3 (8x H200-SXM-141GB, NVIDIA TensorRT) - 3^rd place on 3d-unet-99.9 offline
• ThinkSystem SR680a V3 (8x H200-SXM-141GB, NVIDIA TensorRT) – 3^rd place on gptj-99.9 server

The insights from the latest MLPerf benchmarks are critical for stakeholders in the generative AI and machine learning ecosystem, from system architects to application developers. They provide a quantitative foundation for hardware selection and optimization, crucial for deploying scalable and efficient AI/ML systems. Future developments in hardware and software are anticipated to further influence these benchmarks, continuing the cycle of innovation and evaluation in the field of machine learning.

Professionals in the field are encouraged to consider these results in their future hardware procurement and system design strategies. For further discussion or consultation on leveraging these insights in specific use cases, engage with our expert team at aidiscover@lenovo.com.

For more information, see the following resources:

• Explore Lenovo AI solutions:
  https://www.lenovo.com/us/en/servers-storage/solutions/ai/
• Engage the Lenovo AI Center of Excellence:
  https://lenovo-ai-discover.atlassian.net/servicedesk/customer/portal/3
• MLCommons®, the open engineering consortium and leading force behind MLPerf, has now released new results for MLPerf benchmark suites:
  • Benchmark results: https://mlcommons.org/benchmarks/inference-datacenter/
  • Latest news about MLCommons: https://mlcommons.org/news-blog

David Ellison is the Chief Data Scientist for Lenovo ISG. Through Lenovo’s US and European AI Discover Centers, he leads a team that uses cutting-edge AI techniques to deliver solutions for external customers while internally supporting the overall AI strategy for the World Wide Infrastructure Solutions Group. Before joining Lenovo, he ran an international scientific analysis and equipment company and worked as a Data Scientist for the US Postal Service. Previous to that, he received a PhD in Biomedical Engineering from Johns Hopkins University. He has numerous publications in top tier journals including two in the Proceedings of the National Academy of the Sciences.