High availability virtualization on the IBM System x3850 X5

64 highly available virtual machines on two-node IBM® System x3850 X5 cluster with 4 Intel® Xeon® 8-core processors, 192 GB memory, and Windows® Server 2008 R2 Hyper-V

The IBM System x3850 X5 server is the fifth generation of the Enterprise X-Architecture, delivering innovation with enhanced reliability and availability features to enable optimal performance for databases, enterprise applications, and virtualized environments. Benefits of IBM X5 servers include:

• Increased performance with the next generation Intel Xeon processors
• Memory reliability and availability
• Higher performing databases and faster time to value for database workloads
• Up to eight sockets and 128 DIMMs in a two-chassis configuration for larger databases, virtualization, enterprise, and mission-critical workloads

The IBM System x3850 X5 is shown in Figure 1.


Figure 1. The IBM System x3850 X5

Hyper-V in Microsoft Windows Server 2008 R2 can now support 64 logical processors, a significant increase from the 24 logical processors in the first release of Hyper-V. Each virtual machine can be configured with up to 4 virtual processors and 64 GB of memory per virtual machine. Live migration and Clustered Shared Volumes (CSV) features are now supported with this new release of Hyper-V.

The virtual machines described in this document use high-availability Windows Failover clustering on the host servers. A highly available virtual machine (HA VM) provides maximum uptime and responsiveness to users and flexibility to IT administrators. A running virtual machine can be moved with no interruption of service for load balancing and maintenance operations using Live Migration. In the event of a hardware failure, the virtual machine will be quickly restarted on one of the other cluster nodes by Windows Failover clustering. Microsoft Hyper-V now supports up to 32 virtual machines per node in a Windows failover configuration. On a two-node cluster, as shown in this document, 64 VMs are supported. Customers must plan for adequate capacity for when failover occurs and the VMs are to be moved to the other node in the cluster.

A two-node Windows Server 2008 R2 cluster comprised of two x3850 X5 enterprise servers enables high-density consolidation of workloads on a solution that offers both high availability and high performance. You can create and run 64 virtual machines across a two-node x3850 X5 cluster, with four Intel Xeon 8-core processors and 192 GB memory on each server using 48 4GB DIMMs.

Through virtualizing workloads with Microsoft Hyper-V technology on IBM System x3850 X5 servers, businesses can make the best use of server hardware investments by consolidating multiple server roles as separate virtual machines running on a single physical machine. Running the same hypervisor with fewer physical servers allows your IT administrators to manage more servers than without virtualization. Managing applications within virtual machines separated from the actual hardware results in less downtime. Virtualization also creates greater IT flexibility during the deployment and upgrade processes in both physical and virtual machines.

In addition, businesses can enjoy a lower total cost of ownership (TCO) from reduced hardware requirements by maximizing the resource utilization of each server purchased. Reducing the number of physical servers required to support operations lowers capital acquisition, power consumption, and cooling costs. Infrastructure optimization also simplifies and standardizes IT administration, thus helping to control labor costs. Reduced downtime costs are realized by the ability to easily move and balance workloads across different resources, improve operational agility, and provide flexibility in managing maintenance schedules or responding to today’s dynamic IT demands.

Microsoft Hyper-V is a 64-bit technology that runs as a role in Windows Server 2008 R2, or as a stand-alone, console-based hypervisor in the Hyper-V Server 2008 R2 product (available for download from http://microsoft.com/hvs). Virtual machines can support up to 4 virtual processors and 64 GB of memory, depending on the operating system loaded. Each virtual machine will have its own operating system instance and is completely isolated from the host operating system as well as from other virtual machines. High availability is an option when running the Enterprise or Datacenter Edition of Windows Server 2008 R2 with Microsoft Clustering installed or on the Microsoft Hyper-V Server 2008 R2 with Microsoft Clustering installed. Hyper-V is not supported on Intel Itanium processor–based servers.

Complementing the solution, IBM System Storage provides the high performing, scalable, reliable, available, and flexible midrange storage systems needed to support Microsoft Hyper-V information infrastructure. With the IBM System Storage DS5000, you can enhance the improved efficiency and reduced total cost of ownership in your server infrastructure.

As IBM’s most powerful midrange storage system, the DS5000 is well suited for a virtualized environment that can keep pace with your business growth. Organizations can buy only the capacity needed initially, and can then dynamically upgrade and reconfigure additional capacity and features later to meet changing business requirements, all without any system downtime. The DS5000 provides flexibility intermixing both Fibre Channel and SATA disks for midrange storage systems. The DS5000 delivers class-leading performance and is well-suited for consolidation and virtualization.

 

The advanced features incorporated throughout the IBM X-Architecture portfolio help IBM System x servers and IBM BladeCenter chassis and blades deliver the reliability and high availability necessary to virtualize a wide range of applications on Hyper-V:

• System x3850 X5 enterprise servers offer up to 64 logical processors, up to 1 TB of memory with 64 16GB DIMMs, seven PCI Express 2.0 slots, and up to eight internal HDDs.
• Powered by next generation Intel Xeon processors with up to eight cores.
• Highly available memory with IBM Active Memory Memory ProteXion, Chipkill, and memory mirroring.
• Superior memory per processor capacity supports the needs of virtualization servers in 2010 and beyond.
• Superior memory technology helps to lower cost, lower latency, increase bandwidth, and lower power consumption with MAX5 and the Advanced Buffer eXecution chip.

To test the maximum supported limit of 64 VMs per node in a 2-node x3850 X5 cluster by Microsoft Hyper-V, we set up two IBM x3850 X5 servers, each using the following configuration:

• Four Intel Xeon 8-core processors
• 48 4GB Samsung DIMMs for a total of 192 GB memory
• Two 146GB HDDs in RAID-1 for the operating system
• One Emulex LP12002 8Gb dual-port HBA
• One Broadcom 5709 Dual-Port NIC
• One Intel PRO/1000 PT Dual-port NIC
• One DS5300 dual storage controller with 4GB battery-backed cache
• Four EXP810s with 16 300GB HDDs each for a total of 64 HDDs to host the VMs

Using System Center Virtual Machine Manager (SCVMM) 2008 R2 virtual machine hardware and software profiles were created. SCVMM allows the administrator to set predefined memory size, network assignments, processors, and hard disk in the hardware profile, and Active Directory membership, default password, and license key in the software profile. These profiles were paired with a sysprep’d virtual machine stored in the library to create the 64 virtual machines.

The virtual machines were placed on the attached fault-tolerant DS5000 arrays. The volumes were built using the Clustering Share Volume feature of Microsoft Windows Server 2008 R2. Sizing should take into account not only the raw size of the virtual machines, but also the aggregate I/O requirements expected from the sum of the virtual machines. The greater the disk I/O requirements the more disk spindles need to be committed to the arrays. In addition, consideration should be made to ensure sufficient network I/O with additional NIC(s) for your host server as needed.

Each of the 64 virtual machines was configured as follows:

• Up to four virtual processors
• 1GB to 16GB of virtual memory
• 12 GB virtual hard disk for the operating system
• One virtual network port

For this technology demonstration, there was no specific mission-critical workload being run on the virtual machines and therefore memory, processor, and I/O utilization were fairly low. The memory and I/O capacity of the x3850 X5 server was not maximized and provided more than ample processing power headroom. This proof of concept represented a consolidation with virtualization deployment scenario implemented in a two-node clustered environment for fault tolerance and high availability. The test confirmed that the supported limit of 32 Hyper-V virtual machines per host in a cluster can be easily hosted on a two-node x3850 X5 cluster and demonstrated the capabilities of the IBM X5 architecture and the scalability of Microsoft Hyper-V.

Figure 2 shows a snapshot of the two-node cluster running 64 virtual machines.


Figure 2. VMM overview showing 64 virtual machines running on the two-node cluster

The IBM System x3850 X5 can easily handle even the maximum supported configuration using Microsoft Windows Server 2008 R2 with Hyper-V virtualization and is well-suited to consolidate underutilized Servers in an easy-to-manage highly available configuration. By providing an environment with a reduced number of physical machines to manage and increased availability, the solution greatly improves TCO with reduced space, power, and cooling needs.

The IBM System x3850 X5 server is a high performing, reliable server that continues to set the pace in the x86 market. This server drives the industry to new levels of virtualization densities, availability, flexible scalability, and transaction performance. IBM System x3850 X5 servers maximize memory, minimize cost, and simplify development. Fifth-generation Enterprise X-Architecture technology with the latest Intel Xeon Processors delivers affordable pricing and leadership performance without compromising high availability or rack density. Coupled with high performing, highly available, and flexible IBM System Storage DS5000 systems, an x3850 X5 server and Microsoft Hyper-V solution will help optimize your IT HA environment.

To learn more about virtualization using Microsoft Hyper-V on the IBM System x platform, please visit:

http://ibm.com/systems/x/solutions/infrastructure/virtualization

To learn more about IBM storage solutions, see the paper IBM System Storage Solutions for Microsoft Hyper-V, available from:

ftp://ftp.software.ibm.com/common/ssi/pm/sp/n/tss03028usen/TSS03028USEN.PDF

For more information about IBM eX5 and the IBM System x3850 X5, please visit:

http://ibm.com/systems/ex5

Don Clemmer is an IBM System x systems engineer. For over 12 years, Don has worked at the IBM Center for Microsoft Technologies with IBM customers and partners in designing Windows- and virtualization-based solutions to meet their business needs. He is currently based at Microsoft's Redmond campus working on optimizing the performance of IBM's System x servers running Microsoft Windows Server operating systems environments.

Scott Smith is an IBM System x systems engineer working at the IBM Center for Microsoft Technology. Over the past 15 years, Scott has worked to optimize the performance of IBM x86-based servers running the Microsoft Windows Server operating system and Microsoft application software. He has extensive experience in helping IBM customers to understand the issues they are facing and developing solutions that address them.

Vinay Kulkarni is an IBM System x performance engineer working on-site at the Microsoft Redmond campus. He has been working with Microsoft over the past 8 years to optimize performance of System x servers running Microsoft Windows and SQL Server software. He also works with IBM clients to tune performance of System x and storage environments and supports publication of industry-leading benchmarks alongside System x benchmark teams.