Top
Author
Updated
25 Oct 2024Form Number
LP1693PDF size
16 pages, 219 KBSubscribed to LP1693.
Thank you for your feedback.
Table of Contents
Abstract
The ThinkSystem NVIDIA ConnectX-7 NDR200 InfiniBand QSFP112 Adapters offer 200 Gb/s connectivity for both Ethernet and InfiniBand protocols. The adapters are suitable for providing high-performance connectivity when running HPC, cloud, storage and machine learning applications.
This product guide provides essential presales information to understand the ConnectX-7 NDR200 adapters and their key features, specifications, and compatibility. This guide is intended for technical specialists, sales specialists, sales engineers, IT architects, and other IT professionals who want to learn more about the ConnectX-7 NDR200 adapters and consider their use IT solutions.
Change History
Changes in the October 25, 2024 update:
- Added the following water-cooled adapter:
- ThinkSystem SC750 V4 NVIDIA ConnectX-7 NDR200/HDR/200GbE QSFP112 2-Port PCIe Gen5 x16 Adapter (SharedIO) DWC, 4XC7A99370
Introduction
The ThinkSystem NVIDIA ConnectX-7 NDR200 InfiniBand QSFP112 Adapters offer 200 Gb/s connectivity for both Ethernet and InfiniBand protocols. The adapters are suitable for providing high-performance connectivity when running HPC, cloud, storage and machine learning applications.
The following figure shows the ThinkSystem NVIDIA ConnectX-7 NDR200/200GbE QSFP112 2-port PCIe Gen5 x16 Adapter (the standard heat sink has been removed in this photo).
Figure 1. ThinkSystem NVIDIA ConnectX-7 NDR200/200GbE QSFP112 2-port PCIe Gen5 x16 Adapter
Did you know?
The ConnectX-7 NDR200 adapters are optimized to deliver accelerated networking for modern cloud, artificial intelligence, and traditional enterprise workloads. ConnectX-7 provides a broad set of software-defined, hardware-accelerated networking, storage, and security capabilities which enable organizations to modernize and secure their IT infrastructures.
Part number information
The following table shows the part numbers for the adapters.
* In the configurator, the server-specific copper tubing for water cooling is automatically derived and attached in the factory
Non-DWC adapter part numbers includes the following:
- One NVIDIA adapter with full-height (3U) adapter bracket attached
- Low-profile (2U) adapter bracket
- Documentation
Supported transceivers and cables
The ConnectX-7 NDR200 adapters have two empty QSFP112 cages for connectivity. The following table lists the supported transceivers.
The following table lists the supported transceivers.
The following table lists the supported fiber optic cables and Active Optical Cables.
The following table lists the supported direct-attach copper (DAC) cables.
Features
The adapters have the following features:
- Accelerated Networking and Security
ConnectX-7 provides a broad set of software-defined, hardware-accelerated networking, storage, and security capabilities which enable organizations to modernize and secure their IT infrastructures. Moreover, ConnectX-7 empowers agile and high-performance solutions from edge to core data centers to clouds, all while enhancing network security and reducing the total cost of ownership.
- Accelerate Data-Driven Scientific Computing
ConnectX-7 provides ultra-low latency, extreme throughput, and innovative NVIDIA In-Network Computing engines to deliver the acceleration, scalability, and feature- rich technology needed for today’s modern scientific computing workloads.
- Accelerate Software- Defined Networking
NVIDIA ASAP2 technology accelerates software-defined networking, delivering line-rate performance with no CPU penalty.
- Enhance Storage Performance
ConnectX-7 enables high- performance and efficient data storage by leveraging RDMA/RoCE, GPUDirect Storage, and hardware-based NVMe-oF offload engines.
- SharedIO (4XC7A86669 only)
SharedIO (Shared I/O) is an implementation of Sockets Direct that is offered in Lenovo direct water-cooled (DWC) servers such as the SD650 V3, where there are two server nodes per DWC tray. In this implementation, the ConnectX-7 adapter is installed in a PCIe slot in one node and the adapter is connected via a cable to a PCIe connector on the second node. The result is that the two nodes share the network connection of the adapter with significant savings both in the cost of the adapters but also the cost of switch ports.
Technical specifications
The ConnectX-7 NDR200 adapters have the following technical specifications:
- Networking interfaces
- Two QSFP112 cages for 200 Gb/s connectivity
- 400Gb/s total bandwidth
- InfiniBand connectivity
- Support for InfiniBand NDR200 or HDR with speeds up to 200 Gb/s
- InfiniBand Trade Association Spec 1.5 compliant
- RDMA, send/receive semantics
- 16 million input/output (IO) channels
- 256 to 4Kbyte maximum transmission unit (MTU), 2Gbyte messages
- Enhanced InfiniBand Networking
- Hardware-based reliable transport
- Extended Reliable Connected (XRC)
- Dynamically Connected Transport (DCT)
- GPUDirect RDMA
- GPUDirect Storage
- Adaptive routing support
- Enhanced atomic operations
- Advanced memory mapping, allowing user mode registration (UMR)
- On-demand paging (ODP), including registration-free RDMA memory access
- Enhanced congestion control
- Burst buffer offload
- Single root IO virtualization (SR-IOV)
- Optimized for HPC software libraries including: NVIDIA HPC-X, UCX, UCC, NCCL, OpenMPI, MVAPICH, MPICH, OpenSHMEM, PGAS
- Collective operations offloads
- Support for NVIDIA Scalable Hierarchical Aggregation and Reduction Protocol (SHARP)
- Rendezvous protocol offload
- In-network on-board memory
- Ethernet connectivity
- Supports speeds up to 200 GbE
- Supports NRZ (10G, 25G) and PAM4 (50G, 100G)
- RDMA over Converged Ethernet (RoCE)
- Enhanced Ethernet Networking
- Zero-Touch RoCE
- ASAP² Accelerated Switch and Packet Processing for SDN and VNF acceleration
- Single Root I/O Virtualization (SR-IOV)
- VirtIO acceleration
- Overlay network acceleration: VXLAN, GENEVE, NVGRE
- Programmable flexible parser
- Connection tracking (L4 firewall)
- Flow mirroring, sampling and statistics
- Header rewrite
- Hierarchical QoS
- Stateless TCP offloads
- Storage Accelerations
- Block-level encryption: XTS-AES 256/512-bit key
- NVMe over Fabrics (NVMe-oF)
- NVMe over TCP (NVMe/TCP)
- T10 Data Integrity Field (T10-DIF) signature handover
- SRP, iSER, NFS over RDMA, SMB Direct
- Management and Control
- NC-SI, MCTP over SMBus, and MCTP over PCIe
- PLDM for Monitor and Control DSP0248
- PLDM for Firmware Update DSP0267
- PLDM for Redfish Device Enablement DSP0218
- PLDM for FRU DSP0257
- SPDM DSP0274
- Serial Peripheral Interface (SPI) to flash
- JTAG IEEE 1149.1 and IEEE 1149.6
- Remote Boot
- Remote boot over InfiniBand
- Remote boot over iSCSI
- UEFI
- PXE over InfiniBand and Ethernet
- Cybersecurity
- Platform security: secure boot with hardware root-of-trust, secure firmware update, flash encryption, and device attestation
- PCI Express Interface
- PCIe Gen 5.0 x16 host interface
- Support for PCIe bifurcation
- SharedIO (NVIDIA Multi-Host) supports connection of up to two hosts (on supported water-cooled nodes only)
- Transaction layer packet (TLP) processing hints (TPH)
- PCIe switch Downstream Port Containment (DPC)
- Support for MSI/MSI-X mechanisms
- Advanced error reporting (AER)
- Access Control Service (ACS) for peer-to-peer secure communication
- Process Address Space ID (PASID)
- Address translation services (ATS)
- Support for SR-IOV
NVIDIA Unified Fabric Manager
NVIDIA Unified Fabric Manager (UFM) is InfiniBand networking management software that combines enhanced, real-time network telemetry with fabric visibility and control to support scale-out InfiniBand data centers.
The two offerings available from Lenovo are as follows:
- UFM Telemetry for Real-Time Monitoring
The UFM Telemetry platform provides network validation tools to monitor network performance and conditions, capturing and streaming rich real-time network telemetry information, application workload usage, and system configuration to an on-premises or cloud-based database for further analysis.
- UFM Enterprise for Fabric Visibility and Control
The UFM Enterprise platform combines the benefits of UFM Telemetry with enhanced network monitoring and management. It performs automated network discovery and provisioning, traffic monitoring, and congestion discovery. It also enables job schedule provisioning and integrates with industry-leading job schedulers and cloud and cluster managers, including Slurm and Platform Load Sharing Facility (LSF).
The following table lists the subscription licenses available from Lenovo.
For more information, see the following web page:
https://www.nvidia.com/en-us/networking/infiniband/ufm/
Server support
The following tables list the ThinkSystem servers that are compatible.
Operating system support
The following table indicates which operating systems can be preloaded in the Lenovo factory for CTO server orders where this adapter is included in the server configuration.
Tip: If an OS is listed as "No support" above, but it is listed in one of the support tables below, that means the OS is supported by the adapter, just not available to be preloaded in the Lenovo factory in CTO orders.
The adapters support the operating systems listed in the following tables.
- ThinkSystem NVIDIA ConnectX-7 NDR200/200GbE QSFP112 2-port PCIe Gen5 x16 Adapter, 4XC7A81883
- ThinkSystem SD650 V3 NVIDIA ConnectX-7 NDR200/HDR/200GbE QSFP112 2-Port PCIe Gen5 x16 Adapter (SharedIO) DWC, 4XC7A86669
Tip: These tables are automatically generated based on data from Lenovo ServerProven.
Regulatory approvals
The adapters have the following regulatory approvals:
- Safety: CB / cTUVus / CE
- EMC: CE / FCC / VCCI / ICES / RCM / KC
- RoHS: RoHS Compliant
Operating environment
The adapters have the following operating characteristics:
- Typical power consumption (passive cables): 19.3W
- Maximum power available through QSFP56 port: 5W
- Temperature
- Operational: 0°C to 55°C
- Non-operational: -40°C to 70°C
- Humidity: 90% relative humidity
Warranty
One year limited warranty. When installed in a Lenovo server, the adapter assumes the server’s base warranty and any warranty upgrades.
Related publications
For more information, refer to these documents:
- ConnectX-7 user manual:
https://docs.nvidia.com/networking/display/ConnectX7VPI/Specifications - NVIDIA InfiniBand product page:
https://www.nvidia.com/en-us/networking/infiniband-adapters/ - DCSC configurator
https://dcsc.lenovo.com - Networking Options for ThinkSystem Servers:
https://lenovopress.com/lp0765-networking-options-for-thinksystem-servers - ServerProven compatibility:
https://serverproven.lenovo.com
Trademarks
Lenovo and the Lenovo logo are trademarks or registered trademarks of Lenovo in the United States, other countries, or both. A current list of Lenovo trademarks is available on the Web at https://www.lenovo.com/us/en/legal/copytrade/.
The following terms are trademarks of Lenovo in the United States, other countries, or both:
Lenovo®
ServerProven®
ThinkSystem®
The following terms are trademarks of other companies:
AMD is a trademark of Advanced Micro Devices, Inc.
Intel® and Xeon® are trademarks of Intel Corporation or its subsidiaries.
Linux® is the trademark of Linus Torvalds in the U.S. and other countries.
Microsoft®, Windows Server®, and Windows® are trademarks of Microsoft Corporation in the United States, other countries, or both.
Other company, product, or service names may be trademarks or service marks of others.
Configure and Buy
Please select a locale
Full Change History
Changes in the October 25, 2024 update:
- Added the following water-cooled adapter:
- ThinkSystem SC750 V4 NVIDIA ConnectX-7 NDR200/HDR/200GbE QSFP112 2-Port PCIe Gen5 x16 Adapter (SharedIO) DWC, 4XC7A99370
Changes in the August 16, 2024 update:
- The adapters support PXE boot support, both over Ethernet and InfiniBand - Technical specifications section
Changes in the August 1, 2024 update:
- Added information about factory preload support for operating systems - Operating system support section
Changes in the September 29, 2023 update:
- The adapters support secure boot, however they do not include embedded cryptography functions - Technical specifications section
Changes in the September 15, 2023 update:
- Withdrawn cables are now hidden; click Show Withdrawn Products to view them - Supported transceivers and cables section
Changes in the July 5, 2023 update:
- Added the following transceiver - Supported transceivers and cables section:
- Lenovo 100GBase-SR4 QSFP28 Transceiver, 4TC7A86257
Changes in the June 13, 2023 update:
- The adapters support Ethernet as well as InfiniBand, on both ports
- The air-cooled adapter is orderable in the General Purpose mode of the DCSC configurator
First published: May 2, 2023
Course Detail
Employees Only Content
The content in this document with a is only visible to employees who are logged in. Logon using your Lenovo ITcode and password via Lenovo single-signon (SSO).
The author of the document has determined that this content is classified as Lenovo Internal and should not be normally be made available to people who are not employees or contractors. This includes partners, customers, and competitors. The reasons may vary and you should reach out to the authors of the document for clarification, if needed. Be cautious about sharing this content with others as it may contain sensitive information.
Any visitor to the Lenovo Press web site who is not logged on will not be able to see this employee-only content. This content is excluded from search engine indexes and will not appear in any search results.
For all users, including logged-in employees, this employee-only content does not appear in the PDF version of this document.
This functionality is cookie based. The web site will normally remember your login state between browser sessions, however, if you clear cookies at the end of a session or work in an Incognito/Private browser window, then you will need to log in each time.
If you have any questions about this feature of the Lenovo Press web, please email David Watts at dwatts@lenovo.com.
