Top
Author
Updated
11 May 2022Form Number
LP1559PDF size
11 pages, 420 KBSubscribed to LP1559.
Thank you for your feedback.
Table of Contents
Abstract
The ThinkSystem Mellanox ConnectX-4 Lx 25Gb 2-port Mezz Adapter is a high-performance 25Gb Ethernet adapter suitable for Lenovo Flex System servers. The adapter is for customers who looking for end-to-end 25 Gb Ethernet speeds in their Flex System environment as well as those who want to maintain their existing 10Gb networking infrastructure while preparing for future upgrades to 25Gb network speeds.
This product guide provides essential presales information to understand the ConnectX-4 Lx adapter and its key features, specifications, and compatibility. This guide is intended for technical specialists, sales specialists, sales engineers, IT architects, and other IT professionals who want to learn more about the adapter and consider its use in a Flex System solution.
Change History
Changes in the May 11, 2022 update:
- Added the I/O module support section - Only the NE2552E Ethernet switch is supported with this adapter
Introduction
The ThinkSystem Mellanox ConnectX-4 Lx 25Gb 2-port Mezz Adapter is a high-performance 25Gb Ethernet adapter suitable for Lenovo Flex System servers. The adapter is for customers who looking for end-to-end 25 Gb Ethernet speeds in their Flex System environment as well as those who want to maintain their existing 10Gb networking infrastructure while preparing for future upgrades to 25Gb network speeds.
The adapter has the standard form factor shape for Flex System mezzanine adapters.
Figure 1. ThinkSystem Mellanox ConnectX-4 Lx 25Gb 2-port Mezz Adapter
Did you know?
The ConnectX-4 Lx adapter continues a long tradition of technology available for the Flex System ecosystem. The adapter enables customers to extend their investment in Flex System by offering high-speed connectivity into their networking infrastructure.
Features
The ThinkSystem Mellanox ConnectX-4 Lx 25Gb 2-port Mezz Adapter has the following key features:
- I/O Virtualization
ConnectX-4 Lx EN SR-IOV technology provides dedicated adapter resources and guaranteed isolation and protection for virtual machines (VMs) within the server. I/O virtualization with ConnectX-4 Lx EN gives data center administrators better server utilization while reducing cost, power, and cable complexity, allowing more virtual machines and more tenants on the same hardware.
- Overlay Networks
In order to better scale their networks, data center operators often create overlay networks that carry traffic from individual virtual machines over logical tunnels in encapsulated formats such as NVGRE and VXLAN. While this solves network scalability issues, it hides the TCP packet from the hardware offloading engines, placing higher loads on the host CPU. ConnectX-4 Lx EN effectively addresses this by providing advanced NVGRE, VXLAN and GENEVE hardware offloading engines that encapsulate and de-capsulate the overlay protocol headers, enabling the traditional offloads to be performed on the encapsulated traffic for these and other tunneling protocols (GENEVE, MPLS, QinQ, and so on). With ConnectX-4 Lx EN, data center operators can achieve native performance in the new network architecture.
- RDMA over Converged Ethernet (RoCE)
ConnectX-4 Lx EN supports RoCE specifications delivering low-latency and high- performance over Ethernet networks. Leveraging data center bridging (DCB) capabilities as well as ConnectX-4 Lx EN advanced congestion control hardware mechanisms, RoCE provides efficient low-latency RDMA services over Layer 2 and Layer 3 networks.
- Mellanox PeerDirect
Mellanox PeerDirect communication provides high efficiency RDMA access by eliminating unnecessary internal data copies between components on the PCIe bus (for example, from GPU to CPU), and therefore significantly reduces application run time. ConnectX-4 Lx EN advanced acceleration technology enables higher cluster efficiency and scalability to tens of thousands of nodes.
- Storage Acceleration
Storage applications will see improved performance with the higher bandwidth ConnectX-4 Lx EN delivers. Moreover, standard block and file access protocols can leverage RoCE for high-performance storage access. A consolidated compute and storage network achieves significant cost-performance advantages over multi-fabric networks.
Specifications
The ThinkSystem Mellanox ConnectX-4 Lx 25Gb 2-port Mezz Adapter has the following technical specifications:
- Mellanox ConnectX-4-Lx ASIC
- PCIe 3.0 x16 host interface (standard Flex System compute node I/O adapter connection)
- Two copper 25GBase-KR SERDES interfaces (2 lanes of 25G each)
- Connections to the I/O module bays in the Flex System Enterprise Chassis
- Mezz slot 1 to I/O module bays 1 and 2
- Mezz slot 2 to I/O module bays 3 and 4
- All connections are internal to the Flex chassis; no transceivers or cables are required
- Supports Message Signal Interrupt (MSI-X)
- Support for PXE boot, iSCSI boot and Wake-on-LAN (WOL)
- Networking Features
- Jumbo frames (up to 9600-Byte)
- 802.3x flow control
- Link Aggregation (IEEE 802.1AX-2008)
- Virtual LANs-802.1q VLAN tagging
- Configurable Flow Acceleration
- Congestion Avoidance
- CPU Offloads
- RDMA over Converged Ethernet (RoCE)
- TCP/UDP/IP stateless offload
- LSO, LRO, checksum offload
- RSS (can be done on encapsulated packet), TSS, VLAN insertion/ stripping, Receive flow steering
- Intelligent interrupt coalescence
- Overlay Networks
- Stateless offloads for overlay
- networks and tunneling protocols
- Hardware offload of encapsulation and decapsulation of NVGRE and VXLAN overlay networks
- Hardware-Based I/O Virtualization
- Single Root IOV
- Multi-function per port
- Address translation and protection
- Multiple queues per virtual machine
- Enhanced QoS for vNICs
- VMware NetQueue support
- Virtualization
- SR-IOV: Up to 256 Virtual Functions
- SR-IOV: Up to 8 Physical Functions per port
- Virtualization hierarchies (e.g. NPAR)
- Virtualizing Physical Functions on a physical port
- SR-IOV on every Physical Function
- Ingress and egress QoS levels
- Guaranteed QoS for VMs
- Protocol Support
- OpenMPI, IBM PE, OSU MPI (MVAPICH/2), Intel MPI
- Platform MPI, UPC, Open SHMEM
- TCP/UDP, MPLS, VxLAN, NVGRE,
- GENEVE
- iSER, NFS RDMA, SMB Direct
- uDAPL
IEEE Standards
The adapter supports these IEEE specifications:
- IEEE 802.3by, 25 Gb/s supporting all FEC modes
- IEEE 802.3ae 10 Gigabit Ethernet
- IEEE 802.3az Energy Efficient Ethernet (supports only “Fast-Wake” mode)
- IEEE 802.3ap based auto-negotiation and KR startup
- IEEE 802.3ad, 802.1AX Link Aggregation
- IEEE 802.1Q, 802.1P VLAN tags and priority
- IEEE 802.1Qau (QCN) Congestion Notification
- IEEE 802.1Qaz (ETS)
- IEEE 802.1Qbb (PFC)
- IEEE 802.1Qbg
- IEEE 1588v2
- PCI Express Gen 3.0
The adapter supports these additional specifications:
- IPv4 (RFQ 791)
- IPv6 (RFC 2460)
Server support
The following table lists the compute nodes that support the adapters.
Internal connectivity
The ThinkSystem Mellanox ConnectX-4 Lx 25Gb 2-port Mezz Adapter does not use transceivers or cables for connectivity to switches. Instead, the adapter connects to the Flex System switches and I/O modules installed in the chassis via internal connections.
The following table shows the connections between adapters installed in the compute nodes and the switch bays in the chassis.
The connections between the adapters installed in the compute nodes to the switch bays in the chassis are shown diagrammatically in the following figure. The figure shows half-wide servers (such as the SN550 V2 with two adapters) and full-wide servers (such as the SN850 with four adapters).
Figure 3. Logical layout of the interconnects between adapters and I/O modules
I/O module support
These adapters can be installed in any I/O adapter slot of a supported Flex System compute node. One or two compatible modules must be installed in the corresponding I/O bays in the chassis.
The following table lists the supported I/O modules.
Operating system support
The following table lists the supported operating systems.
Tip: This table is automatically generated based on data from Lenovo ServerProven.
Warranty
The ThinkSystem Mellanox ConnectX-4 Lx 25Gb 2-port Mezz Adapter has a 1-year, customer-replaceable unit (CRU) limited warranty. When installed in a server, the adapter assumes the server’s base warranty and any warranty upgrade purchased for the server.
Physical specifications
The adapters have the following dimensions and weight:
- Width: 100 mm (3.9 in.)
- Depth: 80 mm (3.1 in.)
- Weight: 13 g (0.3 lb)
The adapters have the following shipping dimensions and weight (approximate):
- Height: 58 mm (2.3 in.)
- Width: 229 mm (9.0 in.)
- Depth: 208 mm (8.2 in.)
- Weight: 0.4 kg (0.89 lb)
Regulatory compliance
The ThinkSystem Mellanox ConnectX-4 Lx 25Gb 2-port Mezz Adapter conforms to the following regulatory standards:
- Safety: CB, cTUVus, CE
- EMC: CE, FCC, VCCI, ICES, RCM
- RoHS: RoHS-R6
Related publications and links
For more information, see the following resources:
- ThinkSystem NE2552E Flex Switch Product Guide
http://lenovopress.com/LP0854 - ServerProven
http://www.lenovo.com/us/en/serverproven
Trademarks
Lenovo and the Lenovo logo are trademarks or registered trademarks of Lenovo in the United States, other countries, or both. A current list of Lenovo trademarks is available on the Web at https://www.lenovo.com/us/en/legal/copytrade/.
The following terms are trademarks of Lenovo in the United States, other countries, or both:
Lenovo®
ServerProven®
ThinkSystem®
The following terms are trademarks of other companies:
Intel® and Xeon® are trademarks of Intel Corporation or its subsidiaries.
Linux® is the trademark of Linus Torvalds in the U.S. and other countries.
Microsoft®, Windows Server®, and Windows® are trademarks of Microsoft Corporation in the United States, other countries, or both.
Other company, product, or service names may be trademarks or service marks of others.
Configure and Buy
Please select a locale
Full Change History
Changes in the May 11, 2022 update:
- Added the I/O module support section - Only the NE2552E Ethernet switch is supported with this adapter
First published: February 15, 2022
Course Detail
Employees Only Content
The content in this document with a is only visible to employees who are logged in. Logon using your Lenovo ITcode and password via Lenovo single-signon (SSO).
The author of the document has determined that this content is classified as Lenovo Internal and should not be normally be made available to people who are not employees or contractors. This includes partners, customers, and competitors. The reasons may vary and you should reach out to the authors of the document for clarification, if needed. Be cautious about sharing this content with others as it may contain sensitive information.
Any visitor to the Lenovo Press web site who is not logged on will not be able to see this employee-only content. This content is excluded from search engine indexes and will not appear in any search results.
For all users, including logged-in employees, this employee-only content does not appear in the PDF version of this document.
This functionality is cookie based. The web site will normally remember your login state between browser sessions, however, if you clear cookies at the end of a session or work in an Incognito/Private browser window, then you will need to log in each time.
If you have any questions about this feature of the Lenovo Press web, please email David Watts at dwatts@lenovo.com.
