Introducing the Balanced Profile UEFI setting for Intel Optane Persistent Memory

The Intel Optane Persistent Memory (PMem or DCPMM) provides higher memory capacity support than traditional DRAM which can benefit various applications with massive memory requirements such as virtualization, high-performance computing (HPC), big-data analytics, in-memory databases, Video on Demand (VoD) and SAP HANA.

PMem can be configured in Memory Mode, App Direct Mode, and Mixed Mode (combine Memory and App Direct Mode) to adapt to different user scenarios. Focusing on Memory Mode, this paper elaborates on the low-level access mechanism between DRAM and PMem to explain the difference between the Balanced, Bandwidth Optimized and Latency Optimized profiles for PMem.

Managed by the memory controller inside Intel Xeon Scalable Processor, PMem adopts a Two-Level Memory (2LM) Architecture in Memory Mode as shown in the following figure. Different from a legacy memory hierarchy, the DRAM under the 2LM topology is treated as an L4 cache to store frequently used data, alongside the PMem working as system memory to provide a larger memory capacity compared to DRAM alone for memory hungry applications.

In 2LM mode, DRAM acts as the next level cache (L4), and PMem acts as system main memory. When data is found in DRAM, it is an L4 hit and access latency is the same as the latency to DRAM. However, when data is not found at DRAM, it is an L4 miss, and memory controller has to access data from PMem. This access from PMem, when it happens, incurs higher memory latency.

Figure 1. 2LM Architecture

Based on the 2LM topology, the memory controller can only request data from either DRAM or the PMem at a time, such that the dispatch ratio between DRAM and PMem would be a crucial factor of the system performance. Fine tuning numerous thresholds along with dispatch ratio of memory controller and related parameters in the BIOS, Lenovo introduces a new Balanced Profile for PMem memory mode. Based on internal test results, the performance of both memory bandwidth and latency improved significantly when the Balanced Profile was applied.

With the introduction of Balanced Profile, Lenovo now offer three different PMem preset modes in the BIOS to adapt various kind of workloads, as listed in the following table.

Figure 2. PMem performance mode settings (click the image for a larger version)

OneCLI V2.5.0 or later supports Intel PMem and can also be used to setup the PMem performance mode by using the OneCLI command as shown below:

./OneCli config set IntelOptaneDCPMM.DCPMMPerformanceSetting "Latency Optimized"
./OneCli config set IntelOptaneDCPMM.DCPMMPerformanceSetting "BW Optimized"
./OneCli config set IntelOptaneDCPMM.DCPMMPerformanceSetting "Balanced Profile"

Example out of the OneCLI config set command is shown in the following figure.

Figure 3. OneCLI config set command (click the image for a larger version)

Jimmy Cheng is a performance engineer in the Lenovo Infrastructure Solutions Group (ISG) Performance Laboratory in Taipei, Taiwan. Jimmy joined Lenovo in December 2016. Prior to this, he worked on IBM POWER system assurance and validation, system integration, automation development as well as network performance. Jimmy holds a master’s degree in Electronic and Computer Engineering from National Taiwan University of Science and Technology in Taiwan, and a bachelor’s Degree in Computer Science and Engineering from Yuan-Ze University, Taiwan.

Thanks to the following people for their contributions to this project:

• Joe Jakubowski, Principal Engineer and Performance Architect in the Lenovo ISG Performance Laboratory in Morrisville, NC
• Nathan Pham, Sr. Performance Engineer and Architect in the Lenovo ISG Performance Laboratory in Morrisville, NC
• Arunkumar Krishnamoorthy, Server Performance Development Engineer in the Lenovo ISG Performance Laboratory in Morrisville, NC
• David Watts, Lenovo Press