Top
Author
Updated
1 Aug 2024Form Number
LP1364PDF size
27 pages, 576 KBSubscribed to LP1364.
Thank you for your feedback.
Table of Contents
Abstract
The ThinkSystem Mellanox ConnectX-6 Lx 10/25GbE SFP28 Ethernet Adapters are high performance 25Gb Ethernet network adapters that offer multiple network offloads including RoCE v2, NVMe over Ethernet and Open vSwitch.
This product guide provides essential presales information to understand the adapter and its key features, specifications, and compatibility. This guide is intended for technical specialists, sales specialists, sales engineers, IT architects, and other IT professionals who want to learn more about the ConnectX-6 Lx adapters and consider their use in IT solutions.
Change History
Changes in the August 1, 2024 update:
- Added information about factory preload support for operating systems - Operating system support section
Introduction
The ThinkSystem Mellanox ConnectX-6 Lx 10/25GbE SFP28 Ethernet Adapters from NVIDIA Networking are high performance 25Gb Ethernet network adapters that offer multiple network offloads including RoCE v2, NVMe over Ethernet and Open vSwitch.
The following figure shows the ConnectX-6 Lx 10/25GbE SFP28 2-Port OCP Ethernet Adapter (the standard heat sink has been removed in this photo).
Figure 1. ThinkSystem Mellanox ConnectX-6 Lx 10/25GbE SFP28 2-Port OCP Ethernet Adapter (with heatsink removed)
Did you know?
NVMe storage devices are gaining popularity by offering very fast storage access. The evolving NVMe over Fabric (NVMe-oF) protocol leverages the RDMA connectivity for remote access. ConnectX-6 offers further enhancements by providing NVMe-oF target offloads, enabling very efficient NVMe storage access with no CPU intervention, and thus improving performance and reducing latency.
Part number information
The following table shows the part numbers for the adapters.
The part numbers include the following:
- One adapter
- For the PCIe adapter: Low-profile (2U) and full-height (3U) adapter brackets
- Documentation
The following figure shows the ConnectX-6 Lx 10/25GbE SFP28 2-Port PCIe Ethernet Adapter (the standard heat sink has been removed in this photo).
Figure 2. ThinkSystem Mellanox ConnectX-6 Lx 10/25GbE SFP28 2-Port PCIe Ethernet Adapter (with heatsink removed)
Supported transceivers and cables
The following table lists the supported transceivers.
25Gb transceivers: When installed in this 25Gb Ethernet adapter, 25Gb transceivers are designed to operate at either 25 Gb/s or 10 Gb/s speeds as listed in the description of the transceiver, however the speed also depends on the negotiation with the connected switch. In most configurations, this negotiation is automatic, however in some configurations you may have to manually set the link speed or FEC mode.
The following table lists the supported optical cables.
The following table lists the supported direct-attach copper (DAC) cables.
Features
Suitable solutions include: Enterprise data centers, Cloud-native, Web 2.0, hyperscale, Secured infrastructure, Telco and Network Function Virtualization (NFV)
Highlights
- High performance with line speed message rate of 75Mpps
- Advanced RoCE
- ASAP2 - Accelerated Switching and Packet Processing
- Overlay tunneling accelerations
- Stateful rule checking for connection tracking
- Hardware Root-of-Trust and secure firmware update
- Best-in-class PTP performance
- ODCC compatible
Performance and security
ConnectX-6 Lx adapters deliver scalability, high-performance, advanced security capabilities and accelerated networking with the best total cost of ownership for 25GbE deployments in cloud, telco, and enterprise data centers. The adapters provide two ports of 25GbE, PCIe Gen4 and Gen3 x8 host connectivity, and are available either as a low-profile PCIe or OCP 3.0 form factor.
Best-in-Class SDN acceleration
Mellanox’s ASAP2 - Accelerated Switch and Packet Processing technology offloads the Software-Defined Networking (SDN) data plane to the adapter, accelerating performance and offloading the CPU in virtualized or containerized cloud data centers. Customers can accelerate their data centers with an SR-IOV or VirtIO interface while continuing to enjoy their SDN protocol of choice.
ConnectX-6 Lx ASAP2 rich feature set accelerates public and on-premises enterprise clouds, and boosts communication service providers (CSP) transition to NFV. ASAP2 supports these communication service providers by enabling packet encapsulations, such as MPLS and GTP, along side cloud encapsulations, such as VXLAN, GENEVE and others.
Industry-leading RoCE
Following the Mellanox ConnectX tradition of industry-leading RoCE capabilities, ConnectX-6 Lx enables more scalable, resilient, and easy-to-deploy RoCE solutions – Zero Touch RoCE. ConnectX-6 Lx allows RoCE payloads to run seamlessly on existing networks without requiring network configuration (no PFC, no ECN) for simplified RoCE deployments. ConnectX-6 Lx ensures RoCE resiliency and efficiency at scale.
Secure your infrastructure
The adapters support ASAP2 connection-tracking hardware offload to improve L4 firewall performance.
ConnectX-6 Lx also delivers supply chain protection with hardware Root-of-Trust (RoT) for Secure Boot as well as Secure Firmware Update using RSA cryptography and cloning-protection, via a device-unique key, to guarantee firmware authenticity.
Technical specifications
Physical adapter:
- Available in two form factors, Low profile PCIe adapter or OCP 3.0 SFF
- PCIe 4.0 x8 host interface (backward compatible with PCIe 3.0)
- Two SFP28 cages
Virtualization / Cloud Native:
- Single Root IOV (SR-IOV) and VirtIO acceleration, Up to 512 VFs per port and 8 PFs
- Support VXLAN, NVGRE, GENEVE
- Stateless offloads for overlay tunnels
Mellanox ASAP2:
- SDN acceleration for Bare metal, Virtualization, Containers
- Full hardware offload for OVS data plane
- Flow update through RTE_Flow or TC_Flower
- OpenStack support
- Kubernetes support
- Rich classification engine (L2 to L4)
- Flex-Parser: user defined classification
- Hardware offload for:
- Connection tracking (L4 firewall)
- NAT
- Header rewrite
- Mirroring
- Sampling
- Flow aging
- Hierarchal QoS
- Flow-based statistics
Cyber Security:
- Hardware root-of-trust
- Secure firmware update
- ASAP2 connection-tracking hardware offload
Note: These adapters do not offer IPsec encryption & decryption offload
Stateless Offloads:
- TCP/UDP/IP stateless offload
- LSO, LRO, checksum offload
- Received Side Scaling (RSS) also on encapsulated packet
- Transmit Side Scaling (TSS)
- VLAN and MPLS tag insertion/stripping
- Receive flow steering
Advanced Timing & Synchronization:
- Advanced PTP
- IEEE 1588v2 (any profile)
- PTP Hardware Clock (PHC) (UTC format)
- Line rate hardware timestamp (UTC format)
- Time triggered scheduling
- PTP based packet pacing
- Time based SDN acceleration (ASAP2)
Storage Accelerations:
- NVMe over Fabric offloads for target
- Storage protocols: iSER, NFSoRDMA, SMB Direct, NVMe-oF, and more
RDMA over Converged Ethernet:
- RoCE v1
- RoCE v2
- Zero-Touch RoCE: no ECN, no PFC
- RoCE over overlay networks
- Selective repeat
- GPUDirect
- Dynamically Connected Transport (DCT)
- Burst buffer offload
Management and Control:
- SMBus 2.0
- Network Controller Sideband Interface (NC-SI)
- NC-SI, MCTP over SMBus and MCTP over PCIe - Baseboard Management Controller interface
- PLDM for Monitor and Control DSP0248
- PLDM for Firmware Update DSP026
Remote Boot:
- Remote boot over Ethernet
- Remote boot over iSCSI
- UEFI support for x86 and Arm servers
- PXE boot
IEEE standards:
- IEEE 802.3ae 10 Gigabit Ethernet
- 25/50 Ethernet Consortium 25G and 50G supporting all FEC modes
- IEEE 802.3by 25G supporting all FEC modes
- IEEE 802.3ad, 802.1AX Link Aggregation
- IEEE 802.3az Energy Efficient Ethernet (supports only “Fast-Wake” mode)
- IEEE 802.3ap based auto-negotiation and KR startup
- IEEE 802.1Q, 802.1P VLAN tags and priority
- IEEE 802.1Qaz (ETS)
- IEEE 802.1Qbb (PFC)
- IEEE 802.1Qbg
- IEEE 1588v2
- IEEE 1149.1 and IEEE 1149.6 JTAG
- PCI Express Gen 3.0 and 4.0
- IPv4 (RFQ 791)
- IPv6 (RFC 2460)
Server support
The following tables list the ThinkSystem servers that are compatible.
Operating system support
The following table indicates which operating systems can be preloaded in the Lenovo factory for CTO server orders where this adapter is included in the server configuration.
Tip: If an OS is listed as "No support" above, but it is listed in one of the support tables below, that means the OS is supported by the adapter, just not available to be preloaded in the Lenovo factory in CTO orders.
The adapters support the operating systems listed in the following tables:
- ThinkSystem Mellanox ConnectX-6 Lx 10/25GbE SFP28 2-port OCP Ethernet Adapter, 4XC7A62582
- ThinkSystem Mellanox ConnectX-6 Lx 10/25GbE SFP28 2-port PCIe Ethernet Adapter, 4XC7A62580
Tip: These tables are automatically generated based on data from Lenovo ServerProven.
1 The iboot hard drive can't be identified)
2 Ubuntu don't support SW iSCSI
3 For limitation, please refer Support Tip 104278
4 The iboot hard drive can't be identified
5 IONG-11838 tips #TT1781
1 The iboot hard drive can't be identified)
2 The OS is not supported with EPYC 7003 processors.
3 ISG will not sell/preload this OS, but compatibility and cert only.
1 The iboot hard drive can't be identified)
2 Ubuntu don't support SW iSCSI
3 The iboot hard drive can't be identified
4 IONG-11838 tips #TT1781
1 The iboot hard drive can't be identified)
2 For limitation, please refer Support Tip TT2258
3 The OS is not supported with EPYC 7003 processors.
4 ISG will not sell/preload this OS, but compatibility and cert only.
Physical specifications
Low profile PCIe adapter:
- Width: 96mm (3.8 inches)
- Height (without PCIe bracket): 69 mm (2.7 inches)
OCP 3.0 SFF adapter:
- Width: 76 mm (3 inches)
- Depth: 115 mm (4.5 inches)
Operating environment
Power consumption:
- Typical power (passive cables):
- PCIe 4.0: 11.53 W
- PCIe 3.0: 10.73 W
- Maximum power (passive cables):
- PCIe 4.0: 12.94 W
- PCIe 3.0: 12.14 W
Temperature:
- Operational: 0°C to 55°C
- Non-operational: -40°C to 70°C
Humidity: 90% relative humidity
Warranty
One year limited warranty. When installed in a Lenovo server, this adapter assumes the server’s base warranty and any warranty upgrades.
Related publications
For more information, refer to these documents:
- Networking Options for ThinkSystem Servers:
https://lenovopress.com/lp0765-networking-options-for-thinksystem-servers - ServerProven compatibility
http://www.lenovo.com/us/en/serverproven - NVIDIA product page for ConnectX-6 Ethernet adapters
https://www.nvidia.com/en-us/networking/ethernet-adapters/ - User Manual for ConnectX-6 Lx adapter
https://docs.mellanox.com/display/ConnectX6LxEN/
Trademarks
Lenovo and the Lenovo logo are trademarks or registered trademarks of Lenovo in the United States, other countries, or both. A current list of Lenovo trademarks is available on the Web at https://www.lenovo.com/us/en/legal/copytrade/.
The following terms are trademarks of Lenovo in the United States, other countries, or both:
Lenovo®
ServerProven®
ThinkSystem®
The following terms are trademarks of other companies:
AMD is a trademark of Advanced Micro Devices, Inc.
Intel® and Xeon® are trademarks of Intel Corporation or its subsidiaries.
Linux® is the trademark of Linus Torvalds in the U.S. and other countries.
Microsoft®, Windows Server®, and Windows® are trademarks of Microsoft Corporation in the United States, other countries, or both.
Other company, product, or service names may be trademarks or service marks of others.
Configure and Buy
Please select a locale
Full Change History
Changes in the August 1, 2024 update:
- Added information about factory preload support for operating systems - Operating system support section
Changes in the April 29, 2024 update:
- Added the following transceiver - Supported transceivers and cables section:
- 10GBASE-LR SFP+ Transceiver, 00FE331
Changes in the October 31, 2023 update:
- Added the following transceiver - Supported transceivers and cables section:
- ThinkSystem Finisar Dual Rate 10G/25G SR SFP28 Transceiver, 4TC7A88638
Changes in the September 17, 2023 update:
- Withdrawn cables are now hidden; click Show Withdrawn Products to view them - Supported transceivers and cables section
Changes in the January 13, 2023 update:
- Indicated the speeds that each 25Gb transceiver supports - Supported transceivers and cables section
Changes in the November 8, 2022 update:
- Added the following breakout cables - Supported transceivers and cables section:
- Lenovo 1.5m 100G to 4x25G Breakout SFP28 Breakout DAC Cable, 4Z57A85043
- Lenovo 2m 100G to 4x25G Breakout SFP28 Breakout DAC Cable, 4Z57A85044
Changes in the June 27, 2022 update:
- Added the following adapter
- ThinkSystem SE450 Mellanox ConnectX-6 Lx 10/25GbE SFP28 2-Port OCP Ethernet Adapter, BMHD
- Added support for the following transceiver - Supported transceivers and cables section:
- SFP 1000Base-T (RJ-45) Transceiver, 00FE333
Changes in the December 14, 2021 update:
- Added support for the following transceivers - Supported transceivers and cables section:
- ThinkSystem Accelink 10G SR SFP+ Ethernet transceiver, 4TC7A78615
- Lenovo 25Gb SR SFP28 Ethernet Transceiver, 4M27A67041
Changes in the October 20, 2021 update:
- Added a note that the 25GbE transceivers can operate at 25 Gb/s or 10 Gb/s speeds when installed in a 25GbE adapter - Supported transceivers and cables section
Changes in the February 15, 2021 update:
- The server support tables are now automatically updated - Server support section
Changes in the November 6, 2020 update:
- Added OS support tables for the adapters - Operating system support section
First published: 13 October 2020
Course Detail
Employees Only Content
The content in this document with a is only visible to employees who are logged in. Logon using your Lenovo ITcode and password via Lenovo single-signon (SSO).
The author of the document has determined that this content is classified as Lenovo Internal and should not be normally be made available to people who are not employees or contractors. This includes partners, customers, and competitors. The reasons may vary and you should reach out to the authors of the document for clarification, if needed. Be cautious about sharing this content with others as it may contain sensitive information.
Any visitor to the Lenovo Press web site who is not logged on will not be able to see this employee-only content. This content is excluded from search engine indexes and will not appear in any search results.
For all users, including logged-in employees, this employee-only content does not appear in the PDF version of this document.
This functionality is cookie based. The web site will normally remember your login state between browser sessions, however, if you clear cookies at the end of a session or work in an Incognito/Private browser window, then you will need to log in each time.
If you have any questions about this feature of the Lenovo Press web, please email David Watts at dwatts@lenovo.com.
