Lenovo LLM Sizing Tool

Lenovo Sizing Tool helps you determine the hardware and infrastructure required to deploy large language models for a wide range of AI workloads. Whether you're building a chatbot, Retrieval-Augmented Generation (RAG) application, AI agent, code assistant, or another generative AI solution, this tool helps you estimate the hardware resources needed to meet your performance and scalability requirements.

The Lenovo Sizing Tool includes two sizing tools:

• GPU Sizing (Calculator): Sizes GPU-based deployments for your model and workload and recommends a Lenovo server configuration.
• CPU Sizing: Estimates capacity and cost for running inference on CPU-only Lenovo servers.

Jump down to Using the Sizing Tool to learn more about the tool.

Version 2.0. Click the Full Change History link to see what's new. Use the Subscribe to Updates link to get notified when the tool gets updated, and use the Feedback link to send us comments and suggestions.

The Lenovo Sizing Tool includes two sizing tools:

• GPU Sizing (Calculator): Sizes GPU-based deployments for your model and workload and recommends a Lenovo server configuration.
• CPU Sizing: Estimates capacity and cost for running inference on CPU-only Lenovo servers.

Using the GPU Sizing Tool

Follow these steps to use the GPU Sizing Tool:

1. Select the Use Case. The model and workload settings are prefilled with recommended values. To enter your own parameters, select Custom Configuration and set them under Advanced Parameters.
2. Select the GPU Type
3. Optionally, select any of the following Deployment options:
   • Enable RAG: Adds GPU memory for an embedding model, plus extra CPU, memory, and vector database storage.
   • High Availability: Adds one extra server (N+1), with a minimum of two servers.
   • Enable NVLink: Uses NVLink speeds for GPU-to-GPU communication, including PCIe cards with an NVLink bridge such as the H200 NVL. If this option is not checked, then it is assumed GPU-to-GPU communications is performed via the PCIe bus and the speed of the PCIe bus is used in the calculations
   • Set GPU count manually: Enter a value in Number of GPUs. The tool uses exactly this many GPUs and warns if they can't host the workload.
4. Advanced Parameters (optional): Expand Advanced Parameters to review or change any prefilled values:
   • Model Family: Select the model to deploy. For a model that isn't listed, select Custom Model, enter its HuggingFace model ID in Custom Model Name (for example, mistralai/Mixtral-8x7B-v0.1) and press Enter. The model details are filled in automatically.
   • Precision: Select the numeric precision the model will run at.
   • Input Tokens and Output Tokens: Set the typical prompt and response length.
   • Concurrent Users and Requests per User per Minute: Set the expected load.
   • GPU Overhead (%): Set the share of GPU memory reserved for runtime overhead.
5. Click the Calculate GPU Requirements The results appear on the right side of the page.

The results show the recommended GPU count with a breakdown of VRAM usage, the recommended server and Lenovo Hybrid AI platform where applicable, and the CPU, memory, storage, and networking requirements. The results also include estimated power usage and key performance metrics such as Time to First Token (TTFT), Inter-Token Latency (ITL), throughput, and maximum concurrent users.

Click Download PDF Report to save a copy of the results.

Using the CPU Sizing Tool

Get started with these steps:

1. Select a Business use case
2. Select Model size
3. Enter the number of Concurrent users required.
4. The Advanced assumptions are filled with default values; you can change them or leave as is.
5. Pick the Server-CPU-Users combination from the Choose server/CPU dropdown list.

For each option, the tool shows the supported capacity and the cost per million tokens compared with running the same workload in the cloud.

Use this tool to compare infrastructure options, optimize resource utilization, and make informed decisions when designing scalable and cost-effective AI deployments on Lenovo infrastructure. All estimates are based on analytical models and benchmark data and should be used as guidance for capacity planning rather than guaranteed production performance.

For more information about the CPU Sizing Tool, see the Lenovo Press paper, From CPU Sizing to Token Economics: A Business-First Tool to Enterprise AI Inference on Intel Xeon 6.