Emulex OCe14100B VFA5.2 Adapter Family

The Emulex Virtual Fabric Adapter 5 Generation 2 (VFA5.2) Network Adapter Family, which includes the OCe14104B-NX 10 GbE adapter builds on the foundation of the previous generation of Emulex VFA5 adapters by delivering performance enhancements, including RoCE v2 support and management enhancements with improved support for Lenovo XClarity. These new features help reduce complexity, reduce cost, and improve performance with Lenovo servers.

The following figure shows the Emulex VFA5.2 ML2 Dual Port 10GbE SFP+ Adapter.

Figure 1. Emulex VFA5.2 ML2 Dual Port 10GbE SFP+ Adapter

The Emulex VFA5.2 adapters support three methods to virtualize I/O, out of the box: Virtual Fabric Mode (vNIC1), Switch Independent mode (vNIC2) and Unified Fabric Port (UFP) mode. With Virtual Fabric, up to eight virtual network ports (vNICs) can be created with a single two-port 10 GbE network adapter.

On System x adapters, storage protocols such as iSCSI and FCoE are also supported by Features on Demand upgrades. Using a common hardware infrastructure for Ethernet and SAN and by virtualizing your network adapter, you can reduce your infrastructure capital expense.

The part numbers to order these adapters are listed in the following table. The FCoE/iSCSI upgrade licenses are a Features-on-Demand field upgrade that enables full hardware offloaded converged networking capabilities of the adapters. Adapters 00AG580 and 01CV770 already includes the FCoE/iSCSI license.

The adapter, when shipped as a stand-alone option, includes the following items:

• One Emulex adapter
• An attached full-height (3U) bracket, and a low-profile (2U) bracket that can be substituted if needed
• Documentation flyer

The FCoE/iSCSI license is delivered in the form of an authorization code that you enter on the Features on Demand website to generate an activation key.

The adapters in the Emulex VFA5.2 family each have two empty SFP+ cages that support the transceivers listed in the following table.

The following table lists the supported fiber optic cables and Active Optical Cables.

The following table lists the supported direct-attach copper (DAC) cables.

The VFA5.2 adapters offer virtualized networking, support a converged infrastructure, and improve performance with powerful offload engines. The adapter has the following features and benefits.

Reduce complexity

• Virtual NIC emulation

  The Emulex VFA5.2 supports three NIC virtualization modes right out of the box: Virtual Fabric mode (vNIC1), switch independent mode (vNIC2), and Unified Fabric Port (UFP). With NIC virtualization, each of the two physical ports on the adapter can be logically configured to emulate up to four virtual NIC (vNIC) functions with user-definable bandwidth settings. Additionally, each physical port can simultaneously support a storage protocol (FCoE or iSCSI).

• Common drivers and tools

  You can deploy faster and manage less when you use both Virtual Fabric Adapters (VFAs) and Host Bus Adapters (HBAs) that are developed by Emulex. VFAs and HBAs developed by Emulex use the same installation and configuration process, streamlining the effort to get your server configured and running, saving you valuable time. They also use the same Fibre Channel drivers, reducing time to qualify and manage storage connectivity. With Emulex's OneCommand Manager, you can manage VFAs and HBAs that are developed by Emulex through the data center from a single console.

• Improved support for Lenovo XClarity
  • Supports agent-free management, eliminating platform agents to reduce complexity
  • Supports automatic inventory management to simplify management
  • Supports real-time monitoring, fault detection and alert handling for a more secure managed environment
  • Supports automatic firmware downloads and installation for true bare metal provisioning

Reduce cost

• Multiprotocol support for 10 GbE

  The Emulex VFA5.2 adapter family offers two 10 Gb Ethernet ports and is cost- and performance-optimized for integrated and converged infrastructures. The VFA5.2 offers a “triple play” of converged data, storage, and low-latency RDMA networking on a common Ethernet fabric. The Emulex VFA provides customers with a flexible storage protocol option for running heterogeneous workloads on their increasingly converged infrastructures. Note: the OCe14104B-NX 4-port adapter does not support FCoE and iSCSI storage protocols.

• Power savings

  When compared with previous generation Emulex VFA3 adapters, the Emulex VFA5.2 adapters can save up to 50 watts per server, reducing energy and cooling OPEX through improved storage offloads.

Improve performance

• VXLAN/NVGRE offload technology

  Emulex supports Microsoft’s network virtualization using generic routing encapsulation (NVGRE) and VMware’s virtual extensible LAN (VXLAN). These technologies create more logical LANs (for traffic isolation) that are needed by cloud architectures. Because these protocols increase the processing burden on the server’s CPU, the VFA5.2 has an offload engine specifically designed for processing these tags. The resulting benefit is that cloud providers can leverage the benefits of VXLAN/NVGRE with no reduction in the server’s performance.

• Full hardware storage offloads

  The Emulex VFA5.2 adapters support a hardware offload engine (optionally enabled via Features on Demand upgrade, standard on 00AG580 and 01CV770). This accelerates storage protocol processing, which enables the server’s processing resources to focus on application workloads, thus improving the server’s performance. Note: the OCe14104B-NX 4-port adapter does not support FCoE and iSCSI storage protocols.

• Advanced RDMA capabilities with RoCE v2

  RDMA over Converged Ethernet v2 (RoCE v2) delivers application and storage acceleration through faster I/O operations with support for Windows Server SMB Direct and Linux NFS protocols.

  The VFA5.2 adapters leverage RoCE v2, enabling server-to-server data movement directly between application memory without any CPU involvement. This provides high throughput and data acceleration on a standard Ethernet fabric, without the need for specialized infrastructure or management.

  The adapters support the new InfiniBand Trade Association (IBTA) RoCE v2 specification for Layer 3 routing. RoCE also helps accelerate workloads in the following ways:

  • Capability to deliver a 4x boost in small packet network performance vs. previous generation adapters, which is critical for transaction-heavy workloads
  • Desktop-like experiences for VDI with up to 1.5 million FCoE or iSCSI I/O operations per second (IOPS)
  • Ability to scale Microsoft SQL Server, Exchange, and SharePoint using SMB Direct optimized with RoCE
  • More VDI desktops/server due to up to 18% higher CPU effectiveness (the percentage of server CPU utilization for every 1 Mbps I/O throughput)
  • Superior application performance for VMware hybrid cloud workloads, with up to 129% higher I/O throughput compared to adapters without offload

The following figure shows the ThinkSystem Emulex OCe14104B-NX PCIe 10Gb 4-Port SFP+ Ethernet Adapter.

Figure 3. ThinkSystem Emulex OCe14104B-NX PCIe 10Gb 4-Port SFP+ Ethernet Adapter

Adapter specifications:

• Dual-channel 10 Gbps Ethernet controller
• Based on the Emulex XE102-P2 ASIC
• PCI Express 3.0 host interface
• Line-rate 10 GbE performance
• 2 SFP+ empty cages to support either SFP+ SR or twin-ax copper connections
  • SFP+ SR link is with the SFP+ SR optical module with LC connectors.
  • SFP+ twin-ax copper link is with the SFP+ direct attached copper module/cable.
• Power dissipation: 14.4 W typical
• Form factor:
  • Mezzanine LOM (ML2) form factor (adapters 00AG560 and 01CV770)
  • PCIe low-profile form factor (adapters 00AG570 and 00AG580)
  • PCIe full-height half-length adapter (7ZT7A00493)

Virtualization features

• Virtual Fabric support:
  • Virtual Fabric mode (vNIC1)
  • Switch Independent mode (vNIC2)
  • Unified Fabric Port (UFP) support
• Four NIC partitions per adapter port
• Complies with PCI-SIG specification for SR-IOV
• VXLAN/NVGRE encapsulation and offload
• PCI-SIG Address Translation Service (ATS) v1.0
• Virtual Switch Port Mirroring for Diagnostic purposes
• Virtual Ethernet Bridge (VEB)
• 62 Virtual functions (VF)
• QoS for controlling and monitoring bandwidth that is assigned to and used by virtual entities
• Traffic shaping and QoS across each virtual function (VF) and physical function (PF)
• Message Signal Interrupts (MSI-X) support
• VMware NetQue / VMQ support
• Microsoft VMQ & Dynamic VMQ support in Hyper-V

Ethernet and NIC features

• NDIS 5.2, 6.0, and 6.2 compliant Ethernet functionality
• IPv4/IPv6 TCP, UDP, and Checksum Offload
• IPv4/IPv6 TCP, UDP Receive Side Scaling (RSS)
• IPv4/IPv6 Large Send Offload (LSO)
• Programmable MAC addresses
• 128 MAC/vLAN addresses per port
• Support for HASH-based broadcast frame filters per port
• vLAN insertion and extraction
• 9216 byte Jumbo frame support
• Receive Side Scaling (RSS)
• Filters: MAC and vLAN
• UEFI and legacy PXE boot

Remote Direct Memory Access (RDMA):

• Direct data placement in application buffers without processor intervention
• Supports RDMA over converged Ethernet (RoCE) specifications
  • Supports RoCE v2 by replacing RoCE over IP with RoCE over UDP
  • 20% more RoCE Queue Pairs compared to VFA5
  • 16K RoCE Memory Regions (VFA5 offers 8K)
• Linux Open Fabrics Enterprise Distribution (OFED) Support
• Low latency queues for small packet sends and receives
• Local interprocess communication option by internal VEB switch
• TCP/IP Stack By-Pass

Data Center Bridging / Converged Enhanced Ethernet (DCB/CEE):

• Hardware Offloads of Ethernet TCP/IP
• 802.1Qbb Priority Flow Control (PFC)
• 802.1 Qaz Enhanced Transmission Selection (ETS)
• 802.1 Qaz Data Center Bridging Exchange (DCBX)

Fibre Channel over Ethernet (FCoE) offload (included with 00AG580 and 01CV770; optional via a Features on Demand upgrade for 00AG570 and 00AG560):

• Hardware offloads of Ethernet TCP/IP
• ANSI T11 FC-BB-5 Support
• Programmable Worldwide Name (WWN)
• Support for FIP and FCoE Ether Types
• Supports up to 255 NPIV Interfaces per port
• FCoE Initiator
• Common driver for Emulex Universal CNA and Fibre Channel HBAs
• 255 N_Port ID Virtualization (NPIV) interfaces per port
• Fabric Provided MAC Addressing (FPMA) support
• Up to 4096 concurrent port logins (RPIs) per port
• Up to 2048 active exchanges (XRIs) per port

iSCSI offload (included with 00AG580 and 01CV770; optional via a Features on Demand upgrade for 00AG570 and 00AG560):

• Full iSCSI Protocol Offload
• Header, Data Digest (CRC), and PDU
• Direct data placement of SCSI data
• 2048 Offloaded iSCSI connections
• iSCSI initiator and concurrent initiator /target modes
• Multipath I/O
• OS-neutral INT13 based iSCSIboot and iSCSI crash memory dump support
• RFC 4171 Internet Storage Name Service (iSNS)

The following IEEE standards are supported:

• 802.3-2008 10Gbase Ethernet port
• 802.1Q vLAN
• 802.3x Flow Control with pause Frames
• 802.1 Qbg Edge Virtual Bridging
• 802.1Qaz Enhanced transmission Selection (ETS) Data Center Bridging Capability (DCBX)
• 802.1Qbb Priority Flow Control
• 802.3ad link Aggregation/LACP
• 802.1AB Link Layer Discovery Protocol
• 802.3ae (SR Optics)
• 802.3p (Priority of Service)
• 1588 Time Synchronization Support (802.1as)
• IPV4 (RFC 791)
• IPV6 (RFC 2460)

The following tables list the ThinkSystem servers that are compatible.

The Emulex VFA5.2 adapter family is supported in the System x servers that are listed in the following tables.

The Emulex VFA5.2 adapter family supports three types of virtual NIC (vNIC) operating modes, and a physical NIC (pNIC) operating mode. This support is built into the adapter:

• Virtual Fabric Mode (also known as vNIC1 mode)

  Virtual Fabric Mode works only with RackSwitch™ G8124E, G8264CS, G8264, G8264T top-of-rack switches. In this mode, the Emulex adapter communicates with the switch to obtain vNIC parameters (using DCBX). A special tag is added within each data packet and is later removed by the NIC or switch for each vNIC group to maintain separation of the virtual data paths.

  In vNIC mode, each physical port is divided into four virtual ports for a maximum of eight virtual NICs per adapter. The default bandwidth for each vNIC is 2.5 Gbps. Bandwidth for each vNIC can be configured through the RackSwitch from 100 Mbps to 10 Gbps, up to a total of 10 Gb per physical port. The vNICs can also be configured to have zero bandwidth if you must allocate the available bandwidth to fewer than four vNICs per physical port. In Virtual Fabric Mode, you can change the bandwidth allocations through the switch user interfaces without requiring a reboot of the server.

  vNIC bandwidth allocation and metering are performed by both the switch and the VFA. In such a case, a bidirectional virtual channel of an assigned bandwidth is established between them for every defined vNIC.

• Switch Independent Mode (also known as vNIC2 mode)

  In Switch Independent Mode, the adapter works with any 10 Gb Ethernet switch. Switch Independent Mode offers the same capabilities as Virtual Fabric Mode in terms of the number of vNICs and the bandwidth each can be configured to have. Switch Independent Mode extends the existing customer VLANs to the virtual NIC interfaces. The IEEE 802.1Q VLAN tag is essential to the separation of the vNIC groups by the NIC adapter or driver and the switch. The VLAN tags are added to the packet by the applications or drivers at each endpoint rather than by the switch.

  vNIC bandwidth allocation and metering are performed only by VFA itself. In such a case, a unidirectional virtual channel is established where the bandwidth management is performed only for the outgoing traffic on a VFA side (server-to-switch). The incoming traffic (switch-to-server) uses the all available physical port bandwidth, because there is no metering performed on either the VFA or a switch side.

  In Switch Independent Mode mode, when storage protocols are enabled on the Emulex 10GbE Virtual Fabric Adapters, six vNICs (three per physical port) are Ethernet, and two vNICs (one per physical port) are either iSCSI or FCoE.

• Unified Fabric Port (UFP)

  Unified Fabric Port provides a feature rich solution compared to the original vNIC Virtual Fabric mode. Supported with RackSwitch™ G8272, G8264CS, and G8264 top-of-rack switches. Like Virtual Fabric mode vNIC, UFP allows carving up a single 10 Gb port into four virtual NICs (called vPorts in UFP). UFP also has a number of modes associated with it, including:
  Only one vPort (vPort 2) per physical port can be bound to FCoE. If FCoE is not desired, vPort 2 can be configured for one of the other modes.

  • Tunnel mode: Provides Q-in-Q mode, where the vPort is customer VLAN-independent (very similar to vNIC Virtual Fabric Dedicated Uplink Mode).
  • Trunk mode: Provides a traditional 802.1Q trunk mode (multi-VLAN trunk link) to the virtual NIC (vPort) interface. In other words, host-side tagging.
  • Access mode: Provides a traditional access mode (single untagged VLAN) to the virtual NIC (vPort) interface which is similar to a physical port in access mode.
  • FCoE mode: Provides FCoE functionality to the vPort.
  • Auto-VLAN mode: Auto VLAN creation for Qbg and VMready environments.
• pNIC mode (physical NIC mode)

  In pNIC mode, the adapter operates as a standard dual-port 10 Gbps Ethernet adapter, and it functions with any 10 GbE switch. In pNIC mode, with the Emulex FCoE/iSCSI License, the card operates in a traditional Converged Network Adapter (CNA) mode with two Ethernet ports and two storage ports (iSCSI or FCoE) available to the operating system.

The following table compares the three virtual fabric modes.

VFA 5.2 ML2 adapter:

• Height: 69 mm (2.7 in.)
• Length: 168 mm (6.6 in.)
• Width: 17 mm (0.7 in.)

VFA 5.2 PCIe adapters (low-profile):

• Height: 167 mm (6.6 in)
• Width: 69 mm (2.7 in)
• Depth: 17 mm (0.7 in)

OCe14104B-NX adapter (full-height adapter):

• Length: 167 mm (6.6 in)
• Height: 111 mm (4.4 in)
• Full-height bracket attached

Shipping dimensions and weight (approximate):

• Height: 189 mm (7.51 in.)
• Width: 90 mm (3.54 in.)
• Depth: 38 mm (1.50 in.)
• Weight: 450 g (0.99 lb)

This adapter is supported in the following environment:

• Temperature:
  • Operating: 0 °C to 55 °C (32 °F to 131 °F)
  • Non-operating: -40 °C to 70 °C (-40 °F to 158 °F)
• Humidity: 5 - 95%, non-condensing

One-year limited warranty. When installed in a supported server, these cards assume the system’s base warranty and any warranty upgrade.

The following tables list the supported operating systems for the adapters:

• Emulex VFA5.2 2x10 GbE SFP+ PCIe Adapter, 00AG570
• Emulex VFA5.2 ML2 Dual Port 10GbE SFP+ Adapter, 00AG560
• Emulex VFA5.2 2x10 GbE SFP+ Adapter and FCoE/iSCSI SW, 00AG580
• Emulex VFA5.2 ML2 2x10 GbE SFP+ Adapter and FCoE/iSCSI SW, 01CV770
• ThinkSystem Emulex OCe14104B-NX PCIe 10Gb 4-Port SFP+ Ethernet Adapter, 7ZT7A00493