Mellanox InfiniBand Switches for IBM iDataPlex and IBM Intelligent Cluster

High-performance computing (HPC) solutions require high bandwidth, low latency components to get the highest server efficiency and application productivity. The Mellanox IS5000 family of Director-class and Edge-class InfiniBand switches deliver the I/O performance that meets these requirements. Data centers and cloud computing also require I/O services, such as bandwidth, consolidation and unification, and flexibility, and the Mellanox switches support the necessary LAN and SAN traffic consolidation.

Figure 1 shows the Mellanox IS5200 216-port 40 Gbps InfiniBand Director-class switch.

Figure 1. Mellanox IS5200 216-port 40 Gbps InfiniBand Director-class switch

The IS5000 family of InfiniBand switches delivers one of the highest performance and port density with a complete fabric management solution to enable compute clusters and converged data centers to operate at any scale while reducing operational costs and infrastructure complexity.

Table 1. Ordering part numbers and feature codes

Table 2 lists the transceivers and direct-attach copper (DAC) twin-ax cables that are supported.

Table 2. Support transceivers and DAC cables

The Mellanox IS5000 family of Director-class and Edge-class InfiniBand switches are part of IBM Intelligent Cluster solution (formerly IBM Systems Cluster 1350). IBM Intelligent Cluster is your key to a fully integrated HPC solution. IBM clustering solutions include servers, storage, and industry-leading OEM interconnects that are factory-integrated, fully tested, and delivered to your door, ready to plug into your data center, all with a single point of contact for support.

Mellanox IS5600 648-port 40 Gbps InfiniBand Director-class switch

The IS5600 Director switch provides a high density switching solution, scaling up to 51.8 Tbps of bandwidth in a single enclosure, with latency as low as 100-300 ns port-to-port.. Built with the Mellanox fourth generation InfiniScale IV InfiniBand switch technology, the modular chassis provides the ability to scale clusters near-linearly as the cluster size grows. The IS5600 is a 29U rack mount unit and can hold up to 36 leaf modules. Each leaf module contains 18 40 Gbps ports for a total of up to 648 ports.

The Mellanox IS5600Q-10DNC Switch Bundle contains the following components:

• One IS5600 chassis.
• Capacity for up to 36 leaf modules, with each leaf module containing 18 ports (Feature Code 5445).
• One management module. A second redundant module is supported (Feature Code 5444).
• Eight standard power supplies to power a fully configured chassis. Two additional redundant power supplies are available (Feature Code 5739 or 5748).

Mellanox IS5200 216-port 40 Gbps InfiniBand Director-class switch

Like the IS5600, the IS5200 is a high density switching solution scaling up to 17.28 Tbps of bandwidth in a single enclosure, with latency as low as 100-300 ns port-to-port. With support for up to 12 leaf modules in a single 10U rack-mount chassis, the IS5200 supports up to 216 ports, each operating at speeds of up to 40 Gbps.

The Mellanox IS5200Q-4DNC Switch Bundle contains the following components:

• One IS5200 chassis.
• Capacity for up to 12 leaf modules, with each leaf module containing 18 ports (Feature Code 5445).
• One management module. A second redundant module is supported (Feature Code 5444).
• Three standard power supplies to power a fully configured chassis. An additional redundant power supply is available (Feature Code 5739 or 5748).

Mellanox IS5030 36-port 40 Gbps InfiniBand Edge-class switch

The IS5030 Edge switch provides a high-performing fabric solution in a 1RU form factor by delivering 2.88 Tbps of non-blocking bandwidth with 100 ns port-to-port latency.The IS5030 provides up to 40 Gbps full bidirectional bandwidth per each of the 36 ports. These stand-alone switches are an ideal choice for top-of-rack leaf connectivity or for building small to medium sized clusters. The IS5030 is designed to carry converged LAN and SAN traffic with the combination of assured bandwidth and granular Quality of Service (QoS).

The IS5030 Edge switches are built for high performance, reliability, and manageability. They support redundant and hot-swappable fans and a second power supply can be added for power supply redundancy and hot-swap. They have a built-in PPC405 management module that runs a feature rich embedded chassis management software for hardware health monitoring and diagnostics.

The IS5030Q-1BRC is a compact 1U 16.65-in deep fixed form factor switch, with reversed air flow (connector side to power supply side) and rack-mounting kit tailored for IBM iDataPlex deployments. The Mellanox IS5030Q-1BRC Switch Bundle contains the following components:

• One IS5030 chassis for iDataPlex
• 450W Power Supply (Exhaust) and IS50xx Fan Module (Exhaust)

The IS5030Q-1SFC is a compact 1U 23-in deep fixed form factor switch, with forward air flow (power supply side to connector side) and rack-mounting kit tailored for standard rack deployments.

The Mellanox IS5030Q-1SFC Switch Bundle contains the following components:

• One IS5030 chassis for IBM Intelligent Cluster
• 450W Power Supply (Intake) and IS50xx Fan Module (Intake)

Figure 2. Mellanox IS5600 648-port 40 Gbps InfiniBand Director-class switch

The Mellanox IS5000 family of Director-class and Edge-class InfiniBand switches have the following features:

• Converged fabrics

Inter-processor communication (IPC), application data, or transfers to or from storage easily flow through an IS5000 fabric. Granular QoS and guaranteed bandwidth allocation can be applied per traffic type. This ensures that each type of traffic has the resources needed to ensure the highest application performance.

• Sustained network performance

The IS5000 family of switches enables efficient computing by using static routing, adaptive routing, and advanced congestion management. These features ensure the maximum effective fabric bandwidth by eliminating congestion hot spots.

• World-class design

The IS5000 family of switches is designed for performance, serviceability, energy savings, and high availability. The IS5030 switches are optimized for industry-standard racks and scale-out computing solutions from industry leaders. For greater scale-out requirements, the IS5200 and IS5600 chassis switches are the ideal choice. Common modules are used across the entire IS5000 Director class of switches, such as leafs, spines, management modules, power supplies, and fans, which provides ultimate value and ease of use.

• FabricIT: Comprehensive and integrated fabric management solution for cluster computing, database, storage, and grid computing

FabricIT provides a reliable and scalable management solution for InfiniBand networks that can be deployed as a stand-alone manager on a compute or management node or integrated into the Mellanox IS5000 family of switches. FabricIT includes a comprehensive package for cluster power on, fabric-wide bandwidth and latency verification, and various diagnostic tools to check node-node and node-switch connectivity, provide a cluster topology view, and verify the upper layer protocol.

• Low-latency switching

The IS5000 family of switches offer the lowest switching port-to-port latency, providing the most efficient communication between processing nodes, supporting larger and more scalable fabrics, and providing near-linear application performance scaling.

• Granular QoS for cluster, LAN, and SAN traffic
  
  IPC, LAN, and SAN traffic can be carried over a common InfiniBand fabric, thereby reducing the number of adapters, switch ports, and cables and lowering space, power, equipment, and operational costs. With support for granular QoS over the IS5000 switches, different traffic types can share the same fabric resources at the appropriate QoS levels, thereby maximizing the utilization of the fabric while maintaining traffic isolation with no compromises to performance.

Figure 3 shows the Mellanox IS5030 36-port 40 Gbps InfiniBand Edge-class switch.

Figure 3. Mellanox IS5030 36-port 40 Gbps InfiniBand Edge-class switch

The Mellanox IS5600 and IS5200 Director-class switches have the following common specifications:

• 100 ns to 300 ns switching latency
• Hardware-based routing
• Congestion control
• Quality of Service enforcement
• Temperature sensors and voltage monitors
• 18 QSFP InfiniBand ports per leaf module
• Up to 40 Gbps per QSFP
• Port speed auto-negotiation
• Up to 2W per QSFP for active cable fiber module support
• Full bisectional bandwidth to all ports
• IBTA 1.2 compliant
• Hot-swappable fan trays
• Port and system status LED indicators
• RoHS-5 compliant
• One year warranty

The Mellanox IS5600 Director-class switch has the following unique specifications:

• 51.8 Tbps switching capacity
• A 29U modular chassis
• Supports up to 36 leaf modules
• Eight + two redundant auto-sensing 110/220VAC power supplies

The Mellanox IS5200 Director-class switch has the following unique specifications:

• 17.28 Tbps switching capacity
• A 10U modular chassis
• Supports up to 12 leaf modules
• Three + one redundant auto-sensing 110/220VAC power supplies

The Mellanox IS5030 Edge-class switch has the following specifications:

• 2.88 Tbps switching capacity
• 1U rack mount unit
• 36 ports
• Up to 40 Gbps per port
• Fully non-blocking architecture
• Signal optimization for longer cable length
• Quality of Service enforcement
• Up to six virtual subnets
• Temperature sensors and voltage monitors
• Reversible air flow
• Redundant power supplies
• Replaceable fan drawers
• Fan speed controlled by management software

The Mellanox IS5600 Director-class switch has the following physical specifications:

• Height: 29U (50.75 in)
• Width: 19 in
• Depth: 27 in
• Weight: 725 lb, fully configured

The Mellanox IS5200 Director-class switch has the following physical specifications:

• Height: 10U (17.5 in)
• Width: 19 in
• Depth: 27 in
• Weight: 265 lb

The Mellanox IS5030 Edge-class switch has the following physical specifications:

• Height: 1U (1.75 in)
• Width: 17.17 in
• Depth: 16.7 in (iDataPlex) or 23 in (Standard)
• Weight: 15 lb (1PSU) or 17 lb (2PSU)

The switches are supported in the following environment:

Operating temperature: 10 to 45° C
Humidity: 10% to 90% non-condensing
Altitude: -60 m to 2000 m
Input Power: 110/220 VAC auto-sensing (each power supply)

For more information, refer to these documents:

• Publications for the IBM Intelligent Cluster portfolio:
  http://ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5080504
• IBM Intelligent Cluster home page:
  http://www.ibm.com/systems/x/hardware/cluster/
• Mellanox IS5600 InfiniBand Director-class switch product page:
  http://www.mellanox.com/content/pages.php?pg=products_dyn&product_family=59
• Mellanox IS5200 InfiniBand Director-class switch product page:
  http://www.mellanox.com/content/pages.php?pg=products_dyn&product_family=74
• Mellanox IS5030 InfiniBand Edge-class switch product page:
  http://www.mellanox.com/content/pages.php?pg=products_dyn&product_family=60